Desire2Learn

10.2 Upgrade

Later this week both of our production systems will get upgraded from Desire2Learn 9.4.1 to 10.2. This will be an epic effort. Over several days a few dozen people will work on various aspect of the databases, application servers, and testing. Good ‘ole clean IT fun just in time for Christmas. Finding the magical gap between classes and end of term was hard enough when our upgrades took a day. These multi-day ones are much tougher.

Back at FUSION a coworker and I attended a presentation by the team lead for D2L’s Automation group. Thankfully the excellent work from there will be used as part of the upgrade to identify problems earlier. Also, our client testing appears to take a leap forward to automating testing. (We need to take a light year leap forward with application level monitoring.)

Six months of work will be complete.

Then we get to turn around and do it again for May.

Extend LMS With LTI Tools #USGRockEagle13

David Robinson, Georgia Gwinnett College

  • Mash up nifty tools with an Learning Management System.
  • It is a standard, so a defined protocol achieves application portability. The tool can work with any LMS the same.
  • Terms:
    • Tool Provider : the tool with its packaging.
    • Tool Consumer : the system using the tool, like D2L.
    • Context : the course or another other location with the consumer like the Org in D2L. IMS contexts need to be mapped to D2L contexts.
    • Link : is a link.
    • Roles : various systems call roles different things. The IMS roles need to be mapped to the D2L roles.
  • LTI Secret Sauce :
    • Tool admin provides link, key, and secret
    • D2L admin creates the External Learning Tool which includes the link, key, and secret.
    • Instructor adds the quick link to the tool for the course.
  • User clicks the link. Information sent to the tool provider such as user, context, role, key, and signature. User launched into the remote tool. A Single-Sign On, aka no login again.
  • Macmillan wants the admin to setup a link for every course. Impossible to sustain. McGraw-Hill one link and instructors setups the course by selecting a book.
  • Select where it will be deployed such as every Course Offering under a Course Template.
  • Open in New Window tends to work better than inside an iFrame.
  • Instructor role by default can create LTIs, even duplicates.

Integrating D2L-Drupal Via LTI #USGRockEagle13

Tom Boyle, Kennesaw State University

  • Half Drupal and half Desire2Learn interest.
  • Request to determine how to decrease time it takes for an online student to go from application to orientation to advisement to registration.
  • Orientation took place in Desire2Learn.
  • Script looks for applicants in Banner and creates the user in Bulk User Management. Registrar enrolls them in the course.
  • Student takes the quiz.
  • Learning Tools Interoperability (LTI) is a way to integrate systems parallel to an LMS without the need for separate custom integrations. Custom output module  uses D2L external learning tools and LTI.
  • Drupal link is not world readable and users do not have to login. D2L configuration familiar. Open source allowed to write custom code to output to Banner. Sometimes the user does not click the button within Drupal so nothing gets captured. Maybe they should automatically click the button for them?
  • YAY!! Tom is using his test instance of D2L to demo instead of production. Love it when our clients use our services right.
  • URLs are not usable unless sent over by Desire2Learn ELT. Yay, for security.
  • Added LTI Tool Provider and OAuth-PHP libraries to Drupal.
  • Got to explain to Tom what some of the values he sees in the session variables.

Discussion:

  • Alternative method:
    • Intelligent agent emails an address to inform them the orientation is complete.
  • KSU going to use similar method for putting staff into course(s) for Ethics test. Really anything where Tom needs to get new users into D2L fast.

D2L Tips and Tricks 10.2 #USGRockEagle13

Terri Brown and Louise Fechter, Georgia Southern University

    • Upgraded to Desire2Learn 10.2 in July 2013. (Actually from 9.4.1 to 10.0 earler then later to 10.2.)
    • Support plan: revised training, more documentation, briefed support team, wrote “tips & tricks”.
    • When get calls, walk through the task then follow up with a document. If no doc exists, then that is a trigger to create one.
    • Left navigation panel expandable. Drag and drop such as from Google Drive. (Requires Google App integration.)
    • Kaltura is where they have faculty place multimedia files.
    • Document viewer OK. Not as fully featured as desktop equivalents, so students probably will have to download many documents to use them. Binder not much better?
    • D2L Community site has templates for download. Organized templates into a folder. Then enable them.
    • Discussion “Must post first” option to make the student start a thread before they can respond to others.
    • Rubrics cannot be graded in discussion tool. Use two browsers.
      • Fort Valley: Open rubric in grade item. Grade topic.
    • Posts marked read automatically.
    • Deleted topics can be restored without outside intervention? TOO COOL. Am I out of a job now?
    • Reading view promoted instead of grid view. Maybe grid going to disappear soon?
    • Pager lost sound. Just an icon notification.
    • Dropbox enter grades for no submission so students see the item on their grades and better calculate their grades. Ungraded items not calculated as zero unless change Grades setting. Restore deleted folders. Bulk upload files.
    • Student was able to submit after the end date. Not able to replicate.
    • If due date AND end date set to the same value, then neither date shows in the calendar. Use only an end date. Serious bug.
    • View student progress is new. Graph of logins. Progress meter.
    • Quiz matching type improved. Recalculation requires clicking on the TRASH CAN icon (does not delete) to recalculate.
    • Lessons learned:
      • Set user expectations.
      • Communicate.
      • News widget.

 

Each D2L Site Broke 25 Million Hits

Night School is the last time I posted stats here. So this is probably due.

This Yaketystats graph records the daily hits for each site in green and pink. The gray is a total of the two. I had it go back to April first to show our spring load compared to how we are starting this fall.

Each of our Desire2Learn instances broke 25 million hits. Our schools set their own calendars, so start dates ranged from the 12th to the 26th. The amount of activity surged on start dates for the largest schools. A small bump on Wednesday of last week, normally there is a small fall off on Wednesdays, is due to a smaller school’s first day of classes.

The green line has three very large schools (1st, 2nd, 4th) who all started on the 19th. It peaked at 27.59M on the 20th.

The pink line’s bump on the 26th was due to its only very large school starting then. It peaked at 25.61M on the 27th.

Tuesday’s peak could be the highest we get until finals in early December. Since the schools have a Drop/Add period, the first few days are usually slightly higher activity.

Hits From 2013-APR-01 to 2013-AUG-29
Hits From 2013-APR-01 to 2013-AUG-29

Back in April, I saw:

@kfrisch#D2LRUF Reported that MnSCU’s D2L gets 25 million hits a day! We’re the largest self-hosted client of D2L in the United States.

The largest client we have still uses around 70% Blackboard Vista. Their plan is to be all in D2L come January. That should push the green one over the point it is bigger than MnSCU. But even just the normal increase in usage should have the pink one fairly comparable to MnSCU.

We should do something like about a billion hits a month.

I would prefer using a pageview metric. Maybe one of these days when I can make better progress on the to-do list.

Host Hopping Cookies

It started with a tweet on Saturday morning.

@MsIngalls: When I go to the Checklist in @Desire2Learn -when I am logged in – I get an error message that says my log in has expired – ideas? #d2l

This sounded like an issue we had in WebCT Vista product I called Failed Sessions. FSes occur when the user is actively working in the product and suddenly gets dumped to the login page. Not to be confused with Login Loop which is providing the correct username and password but never getting a valid session. I hated working either issue because they were never repeatable. The problem could involve any piece of software that could somehow touch a cookie on the user’s computer. Occasionally they were the fault of Weblogic too.

I recommended Amy open a Desire2Learn ticket through our portal on behalf of the professor. I also started my own investigation.

First, I poked around in the logging database for errors involving checklist. I found different courses not the one involved.

Second, I pulled out of our load balancer logs the id number for the course. That yielded plenty of data showing the problem.

These, I added to the ticket and suggested capturing the HTTP headers should the issue not be repeatable by others. Of course, the support agent was not able to repeat it. The headers clearly showed the cookies were not sent.

The professor of course poked holes all through my suggestions of tracking down which of the many software is involved. Different software, hardware, networks, and browsers meant the cause was probably not something residing on the computers. But the issue definitely was still all of these browsers in a wide variety of places all chose not to send the cookies. This is also when he dropped the next bomb that the problem only occurs on links in a specific widget.

Checking the code behind the widget, I only saw simple absolute URLs. Which made me shudder because earlier this week absolute URLs in the login page for a development site put me in production without me being aware for several minutes.

PSA: Only use absolute URLs when sending a visitor to another web server. Say you are here, at www.ezrasf.com and you want someone to see another of my blog entries. Drop http://www.ezrasf.com from the URL and start the path with / (a relative URL). Should I change the host name to blog.ezrasf.com or www.ezrafreelove.com, then the link has better success of working.

It turns out the problem is the professor used the pre-production host name for the web application. The widget absolute URL links used a different host name for production. Both resolve to the same servers. But cookies are tied to a specific host name. So being logged into one of host and getting a link to the variant means the session is not valid at the variant.

At least the workaround and fix are easy.

The workaround for the professor is to stop using the pre-production URL.

The fix is for the widget designer to turn the absolute URLs to relative URLs since they point to same location.

Also, it would be nice for a better error message than:

No Login

Either you have failed to login, or your login has expired.

First the comma is bad grammar. Second, if I am a normal user who encounters this problem, then what can I do to fix it myself? This is not an error someone sees if their password or username are wrong. This is also not what a user normally sees when they are idle too long. But then again, there are lots and lots of potential causes and solutions.

LMS Brackets

I don’t really follow basketball. So, it is odd that I even registered this tweet from Dennis Kavelman, @dkavelman:

Dennis Kavelman  Team D2L has 4 schools in sweet sixteen! Go #marquette, #arizona, #msu, #osu! @desire2learn

He is excited because four universities who are clients of his company are in the major NCAA basketball championship contest. If only four are Deseire2Learn customers, then that means the other twelve are not. That made me wonder what are the Learning Management Systems used by these sixteen schools?

    1. Louisville : Blackboard
    2. Oregon : Blackboard
    3. Michigan State : D2L
    4. Duke : Sakai
    5. Wichita State : Blackboard
    6. La Salle : Blackboard
    7. Arizona : D2L
    8. Ohio State : D2L
    9. Kansas : Blackboard
    10. Michigan : Sakai
    11. Florida : Sakai
    12. Florida Gulf Coast : Angel (Blackboard) and Canvas in 39 days
    13. Indiana : Sakai
    14. Syracuse : Blackboard
    15. Marquette : D2L
    16. Miami (FL) : Blackboard

Looks like the breakdown is:

Blackboard 8
Desire2Learn: 4
Sakai: 4

This is an interesting grouping. I kind of knew Sakai tended to be the product of choice for well off schools with the money to spend on customization. So, schools with strong athletics probably are more likely to have something like Sakai. Of course, I expect Canvas to be better represented too as it is hot of late. While Moodle tends to be favored by really small schools without a budget, I still figured it would have some representation (really just FGCU).

Which LMSes will be involved with those in the Final Four?

Content Migration Progress Tracking

When moving hundreds of thousands of courses between WebCT Vista and Desire2Learn, keeping track of what made it through which stage seems like an obvious hindsight thing to do. I added that last bit because we started to notice where things fell between the cracks starting to pile up. The basic process…

    1. Through Oracle SQL we populate a middleware database with those courses which meet acceptable criteria.
      CRACK: With the wrong criteria, courses are not even selected.
    2. Through Oracle SQL we generate XML listing in 50 count sets of the courses.
      CRACK: A subset of data loaded into the database may be extracted.
    3. A shell script automates running the WebCT command-line backup process to create a package for each course.
      CRACK: The command-line backup fails on some courses.
    4. Desire2Learn scripts pick up the files and convert WebCT formatted packages to Desire2Learn.
      CRACKS: Too big fail. Too long paths fail. This step can fail to create CSV files for the next step.
    5. Converted packages are placed in a queue to be imported into Desire2Learn.
      CRACKS: WebCT Vista courses can have 1,000 characters in the name and D2L fails if there are more than 50. Courses named the same as a previously loaded one but with a different file name loads both into the same course.

So, there are apparently five different stages and eight potential failures to track and no end-to-end tracking to even know what is missing. Which means inventing something to check logs for errors. 

First thing, I tried writing SQL to create an ordered list of the courses that are available.

The WebCT backups were a little tougher to convert into a useful list. The file names follow the format of Course_Parent_1234567890.bak. They were also in various directories, so I ended up doing something like this to get a list of the files, strip off the parents & time stamps, strip off the directories, and order it.

ls */*.bak | awk -F_ ‘{print $1}’ | awk -F\/ ‘{print $2}’ | sort

So I have two ordered lists. Anything in the D2L one and not in the WebCT one ought to be the work of clients making their own stuff. Anything in the WebCT one and not in the D2L one ought to be my missing ones. Only almost every line is a mismatch.

Visually comparing them, I realized the same courses had in effect different names. All the spaces were converted to underscores. So I ended up adding a sed to convert spaces to underscores after the sort.

Then I wrote some more scripts.

    • Go through the logs from #4 and #5 to display the errors. With it I was able to compare my list of missing with the errors and confirm why they did not come through.
    • Some of the cracks can be addressed by making new import files. So I wrote a script to add those lines to a redo.csv file. Touch up the problems and go.

Basically at this point it only covers 3-5. At some point I am going to have to check steps 1-5. 

OrgCode Duplicate Filter

I was asked to work my “Unix magic”.

The problem? Duplicate courses were spooled and converted from the WebCT format to the Desire2Learn. The conversion process creates an import file using the WebCT SourcedId.Id as the OrgCode. The first time the OrgCode is used, it creates a course. The next and subsequent times, it duplicates content. So these duplicate converted courses gave us a situation where we were screwed.

Fortunately our partners at Desire2Learn intercepted the problem before it got worse.

Out of 1,505 still to be imported, there were 468 duplicates. Yes, 31% duplicates.

D2L asked me to filter the imports to remove the duplicates. I said I am too much of a n00b with Windows to pull it off.

The reply was to use Unix.

Boy do I love Bash shell scripting. In two hours I solved it, though after the high of solving something I had no idea how to write this morning in two hours, there must be something wrong with it.

First, my general idea was to read the file line by line and write those lines with OrgCodes that do not yet exist to a filtered.csv file. I started out looking to exactly duplicate my existing file in another file by reading it line by line.

A while loop which reads each line and records the whole line in a variable.

INPUTFILE=/path/to/file.txt
exec<$INPUTFILE
while read LINE
do
     stuff...
done

I quickly discovered though that since Windows uses the backslash, that foiled the ability of echo to exactly write every line to a file. The backslash escapes the next character. Neither double nor single quotes helped the situation. Oops. So I decided to use sed to make a temporary copy to duplicate the backslashes. A first backslash escapes the next character, in this case a second backslash.

sed -e 's|\\|\\\\|g'

As an error check, the last thing the script does is a diff -u to compare source and new files. At this stage nothing means perfect. I like the -u to give me easier to read results.

So I was able to get an exact copy of the original. All that was next was to get the OrgCode, check it against my filtered file, and if it did not exist, then add it to the end of the filtered file.

ORGCODE=`cat $LINE | awk -F, '{print $1}'`
IF_EXISTS=`grep $ORGCODE $FILTERFILE`
if [ -z IF_EXISTS ] ; then
     echo $LINE >>  $FILTERFILE
fi

Easy. Too easy?

The checks against my work confirmed it worked.

    1. A sorted version of the source run through this and compared in diff -u consistently showed the correct lines were excluded.
    2. Counts for the number of duplicates and the difference of lines missing works.
    3. A check for the number of duplicate OrgCodes returns nothing on the filtered file.

One of Many

The Learning Management System (LMS) has been a despised technology by some ever since I started working with one, WebCT, in 1999. At the time it was deemed crappy technology that had to improve or die. So today in 2012, about 13 years later, I have to roll my eyes at the pundits writing about how the current technology has not significantly changed in a decade (really more than a decade) because it still offers the same tools and will die unless it adapts.

My first few years, 2006-2010, of working at GeorgiaVIEW, our active user counts doubled every 1.5 years. We plateaued at around 290,000 and grow a few thousand a year. Numbers of actions in the system still doubles every 1.5 year. That is insane growth. Growth unlikely fueled by people despising use of the tool. Right now, we are getting pressure to migrate Summer 2012 content for the Fall 2012 start in Desire2Learn1 because instructors roll over the classes from term-to-term. That speaks of long term consistent loyal use not occasional only as little as have to use. For something on the verge of death, it is hard enough keeping the users happy.

I am a database administrator not a faculty member (or dean or vice president for academic affairs or provost). It seems to me though no one would say, “When you teach a class, the white board in the room is the only tool you can use.” Instead, the push would be to add to the available tools in a neverending pursuit of finding better ones. So we see pressures to integrate the LMS with a variety of similar specialized services. Many are textbook replacements or supplementary services designed specifically for student needs. Others are social media. More and more the LMS is just a portal: a place to organize where students really go to learn.

Also, as an IT guy, I think it is important to have a plan B. Things sometimes fail. As a student I was always annoyed when the instructor had to leave the room for 20% of the class to go track down a piece of chalk because the remaining ones were too small to write. I applauded once in my junior year because the instructor happened to have a piece of chalk in her purse just for that contingency. Similarly, faculty members and even students should think about what to do when the LMS is not there. Heck, what should they do if everything the university IT runs like the web sites, email, portal, and network all disappear. It can happen.

When the university bureaucracy selects and administrates a tool, they will adhere to university policy which adheres to higher education laws. When a faculty member selects and administrates a tool, they should do the same. Unfortunately, that means the faculty member becoming familiar with policy and law. Another challenge is running into different interpretations. An example: a user following @VSUENGL1101 on Twitter could be reasonably expected to be a student at Valdosta State University enrolled in the subject English class 1101. Some say that violates the Family Education Rights and Privacy Act. Some disagree, so it is being debated. The law is old and did not likely anticipate social media, so naturally there is movement towards an update.

I doubt the LMS will simply die because there is something better. Instead it will remain one of many tools for years to come. Like the land line, television, JavaScript, still camera, WiFi, non-smartphone, and (God forbid) pagers.

Note 1: Desire2Learn objects to their product being called an LMS. They prefer Learning Environment on the grounds it integrates with so many other tools.

P.S. This totally is from a sustaining technology perspective. Guess I should write this from a disruptive technology perspective.