Both think the problem must be the other’s fault. Because you know… Their product is perfect. They never have design issues. Ever. So it must be someone else’s fault.

Problem 1:

Upon loading the zip package into the CE/Vista SCORM module, the end user saw:

The SCORM package is not compliant with SCORM Ver 1.2 Conformance Requirements. All supporting schemas must be at the root of the package; the following schemas were not found at the root: [adlcp_rootv1p2.xsd, imscp_rootv1p1p2.xsd, imsmd_rootv1p2p1.xsd] As a result, the package may not perform as desired. Are you sure you want to continue?

I actually feel bad for not immediately recognizing this meant these XSD files needed to be in the zip file.  Blackboard ought to feel bad for having the response this means the imsmanifest.xml in the file was written for a more current version of SCORM than the rather ancient 1.2. I could understand that response if specific items in the file are only relevant in the current version. I don’t understand that response when the same file which works with Vista 3 (created about the same time as SCORM 1.2) works and Vista 8 doesn’t. Sounds more like something changed in Vista 8 to make it more strict.

Fixing the missing XSDs files resulted in a new error:

The SCORM package could not be imported because it does not comply with one or more specifications within the package. The following error was produced: **Parsing Error** Line: 7 Message: cvc-complex-type.2.4.c: The matching wildcard is strict, but no declaration can be found for element ‘lom’. Please inform the SCORM vendor and try again once the problem has been resolved.

Is it wrong to be excited about an error? Dropping the problematic items from the imsmanifest.xml file produces new errors. After five iterations, I don’t seem to be making much progress.

Problem 2:

This SCORM module simply passes to an HTML file with JavaScript some variables to send the user off to the QuestionMark site. It should not this big of a deal.

Somehow Vista 8 is calling the file where it doesn’t exist for the QuestionMark SCORM module but not other modules.

QuestionMark addres:

/webct/scorm/viewer/perceptionSCO.htm?call=scorm&
session=9999999999999999&
href=https://ondemand.questionmark.com/delivery/session.php&
lang=-&customerid=mcg

Known good SCORM module address:

/webct/RelativeResourceManager/999999999999/filename.html

The number after RelativeResourceManager typically can be found under “View File Information” in the file manager. The file name after the number is the name of a file in the zip. Copying the address for the known good let me view it. At this point it seemed logical I could just build the address to the QuestionMark zip manually and see it as a designer (maybe not as a student). Unfortunately, this gives me system exception errors.

Last Solutions:

In trying to solve two problems with one stone, I took the imsmanifest.xml for the known good SCORM module and just changed the href= for resource and file to use the perceptionSCO.html file name for QuestionMark. Still failed to find the file.

BUUUUUUUUUUUUUUUT…

I removed all the variables after the perceptionSCO.htm. Now Vista finds the file. It gives me errors about not having the values in those variables, but it found it.

This is stupid.

The possibilities are:

1. Browser: Use a different one which doesn’t complain.
2. Settings: When it comes to security settings, I’m hesitant to recommend thousands of users change them without being extremely sure doing so is safe. Seems to me having it ignore the issue exposes users to danger. Rumor is also Windows Updates can revert the settings back to defaults.
3. Content: Change how the content is delivered to avoid the issue.

The content is being put inside HTML files rather than using the Web Link tool to open a new window with the video. Even using a Web Link tool to show the content inside the same window causes the IE complaint.

All three will cause a ton of work to address the issues. In my opinion wrangling designers ought to require the least amount of work of the three. Though I guess that would depend on the popularity of IE and neediness of the users.

Say what? We had over 22,000 active sections last term? We have 40 institutions and dozens of professional designers and thousands of faculty designing their own classes. Cryptic emails about being frustrated with Vista means I have to contact the sender, find out what institution, class, problem happened and send that to an institution person who has to figure out which designer to direct it. I’m too far removed to be useful for these requests.

Solutions:

1. CE/Vista ought to keep a broken link report. It would record to a log in the Designer Tools reporting which addresses resulted in errors and a list of the referrers, how many times requests came from each referrer. This gives designers something they can use to address the problems. Maybe give the designers the ability to reset the log on a page when they think they have fixed it. Give them an icon in the class list to see when there are unhandled cases.
2. Assuming the above is too hard, this mailto link should be a form recording the learning context involved, referring page, browser, user, and file which failed the request. The message could go to the mail tool for the designers.

Mitigated speech gets a lot of use by people trying not to offend. All too often, people who have been hurt because of mitigated speech question what isn’t being told as though the omission or gaps are intentionally deceptive.

What are or are not supported browsers came up again. The trick here is the mitigated speech used with the levels of support. I assume the intent is clarity.

• Certified – supported with complete testing done.
• Compatible – supported with some testing done.
• Provisional – supported with some testing done before official release.

Certified is taken as supported by all parties. Compatible and Provisional are interpreted as not supported because the complete testing has yet to be done. I think Blackboard’s intent was to mark them as supported but qualify how customers might encounter issues due to not fully testing. This means Blackboard is interested in learning about the problems encountered in order to address them.

At least that is my interpolation. Mmmmmm the Kool-Aid is good.

siapi output says:

error invoking method in adapter, message is: cvc-complex-type.2.3: Element ‘extension’ cannot have character [children], because the type’s content type is element-only.

Means the wrong type between luminis and ims was used. I created a files.properties file which sets the import type based on the name of the file. For the luminis type, the persons records are in persons1.xml. For the ims type, the persons records are in ims_users1.xml.

Message is : Authorization denied

User trying to import the file must be an institution administrator. I probably created the import user but did not enroll it as an admin. What I get for moving too fast.

This means we frequently get oddities in the files which cause problems. Thankfully I am not the person who has to verify these files are good. I just get to answer the questions that person has about why a particular file failed to load.

The CE/Vista import process will stop if its validator finds invalid XML. Unfortunately, the error “An exception occurred while obtaining error messages.  See webct.log” doesn’t sound like invalid XML.

Usage is pretty simple:

xmllint –valid /path/to/file.xml | head

1. If the file is valid, then the whole file is in the output.
2. If there are warnings, then they precede the whole file.
3. If there are errors, then only the errors are displayed.

I use head here because our files can be up to 15MB, so this prevents the whole file from going on the screen for the first two situations.

I discovered this in researching how to handle the first situation below. It came up again today. So this has been useful to catch errors in the client supplied files where the file failed to load.

1: parser error : XML declaration allowed only at the start of the document
 <?xml version=”1.0″ encoding=”UTF-8″?>

162: parser error : EntityRef: expecting ‘;’
<long>College of Engineering &amp&#059; CIS</long>

(Bolded the errors.) The number before the colon is the line number. The carat it uses to indicate where on the line an error occurred isn’t accurate, so I ignore it.

My hope is to get this integrated into our processes to validate these files before they are loaded and save ourselves headaches the next morning.

The first time this came up, Blackboard support wanted us to overhaul the session management. BIG-IP documents saying attempting this new method was a horrible idea caused us never to get on board. We agreed to conduct dupe.pl tests which showed there wasn’t a problem with session spray, which the solution was designed to resolve. Stonewalled, we closed the ticket when the institution reporting it didn’t have any cases to provide us.

So our client with the issue asked us to resume work on it. The key information they provided me was their users hit the /webct/logonDisplay.dowebct. Since they use Single-Sign On (SSO) from a portal, no users should ever hit this page. From investigating these cases, I was able to find a number of cases of users hitting /webct/displayAssessment.dowebct or /webct/displayAssessmentIntro.dowebct with the guest user.

See, the guest user exists at the domain learning context. Users appear as guest before they login or as the logout. They should not appear as guest when taking a quiz.

So I provided this information to Blackboard with the web server logs. They wanted more cases, so I provided more. More clients reported the issue, so I had plenty of sources. Plus it pointed to this problem affecting at least 4 if not all clusters.

Next, our TSM left, so we were provide a new person unused to us. It took just the first note to make a huge mistake. “Provide us all the logs from all the nodes.” At 5GB of logs times 14 nodes in a cluster, 70GB of information for an event which took up maybe 10KB seems like overkill. So… No. I like to think of my self as proficient at system administration, which means I can gather whatever logs you desire.

Now we come to the second mistake. Please refrain from asking me questions already explained in the ticket. Sure, the ticket has a large amount of information. However, if I can remember what is in the ticket, then so can the people working it.

Unfortunately I had to answer a question about replicating this with: it was based on my log trolling not actual cases of students complaining. My mistake was not going to the clients to find a description of the problem. Therefore, Blackboard wanted a WebEx so I could explain the same one sentence repetitively. *headesk* We agreed on me getting a case where a user could explain the problem.

As luck would have it, I got just a case a few days later. So I captured the web server log information and sent it along with the user description. My laziness resulted in me not trimming the log set down to the period of the error. Therefore, this log set showed a user1 login, user2 login, then user1 login again. Blackboard responded this might be a case of sporadic shifting users. Hello! I guess these folks are not used to seeing the SSO login to be able to know the session shifted to another user because… it… logged… in?

By pulling the entries from the f5 log showing the client IP address, Blackboard now wants us to implement a configuration change to the f5 to reflect the browser’s IP in our web server log. Getting such a change isn’t easy for us. Don’t say this is the only way to get client IPs when I… have… sent… you… client IPs. We’ve been at this impasse for 3 weeks. So I get to have another WebEx where I explain the same thing I’ve already written. *headesk*

Maybe it is finally time to ask the people if they are at all familiar with the known issue which sounds like the issue?

VST-3898: When taking an assessment the session is not kept alive. The student’s session times out forcing the student to restart the assessment or makes them unable to complete the assessment.

We plan to implement the upgrade which resolves this issue next week. So, I am hoping this does resolve it. Also, I am tempted to just close this ticket. Should the institutions find they are still having problems in January when the students have had a few quizzes fail, then I might have forgotten how utterly completely useless Blackboard has been on this issue.

All I ask is:

1. Know the information in the ticket so I don’t have to copy and paste from the same ticket.
2. Don’t ask for all the logs. Tell me what logs you want to view.
3. Don’t tell me something is the only way when I’ve already shown you another way. I’m not an idiot.
4. Don’t ask me if the f5 log has the cookie when the entries I’ve already sent you don’t have it.

Tip: Keep a ‘last known good’ config.xml file on your admin node.
Your Bb Vista cluster config.xml file | laura gekeler

Very good tip! We learned this lesson a couple months into hosting Vista 8 in pre-production.

She goes on to explain this file gets overwritten any time a change in Weblogic is made. Sometimes these changes are (un)intentionally made by an administrator using the console. Sometimes Weblogic detects problematic conditions and makes the change itself. When these changes create problematic conditions, then the last known good version saved us from having to go make the changes ourselves and potentially miss something.

In our cases of problems, a single node failing to shutdown after the JMS node was shutdown caused Weblogic to rewrite who should be the JMS node. It also caused a jumbling of the user preferred server stanzas. We now monitor for these problems and page ourselves to warn us about the problem so we can address it immediately rather than let our clients discover the problem a few days later. (I also somewhat mentioned this in the Forcing Weblogic’s Config.xml post.)

The times I’ve done this I didn’t go as far as Laura. I just…

1. Stop all the nodes in the cluster.
2. Copy the current version from $WEBLOGIC_DOMAIN/config/config.xml to /home/<user>.
3. Copy the last known version from /pathto/lastknownversion/config.xml to $WEBLOGIC_DOMAIN/config/.
4. Use “touch $WEBLOGIC_DOMAIN/REFRESH” on each managed node.

I am now curious about why she wanted new server directories or a new Vista_WLSstore database table?

In my opinion this situation arises because Microsoft makes it difficult to notice a shortcut file isn’t the actual file on Windows.

1. By default Windows XP and Vista (anyone know about Win 7?) have “Hide extensions for known file types” checked. So a student is unlikely to notice a shortcut has .lnk instead of .docx by looking at the file name. I think this situation was created to solve the issue of users renaming files to the wrong extension arbitrarily. Windows depends on the file extension for much software.
2. Most file icons have a set appearance. Shortcut icons look like the target file only with a small white box with an arrow. Most users are likely to miss this distinction in my opinion.

Our recommendation to students is to remove the setting mentioned in #1. This ought to better help students figure out which file they are going to submit. Let’s hope, anyway.

Followers of this blog may recall I changed the Weblogic config.xml to record user agents to the webserver.log.

One trick I use is the double quotes in awk to identify just the user agent. This information is then sorting by name to count (uniq -c) how many of each is present. Finally, I sort again by number with the largest at the top to see which are the most common.

grep <term> webserver.log | awk -F\” ‘{print $2}’ | sort | uniq -c | sort -n -r

This is what I will use looking for a specific user. If I am looking at a wider range, such as the user age for hits on a page, then I probably will use the head command to look at the top 20.

A “feature” of this is getting the build (Firefox 3.011) rather than just the version (Firefox 3). For getting the version, I tend to use something more like this to count the found version out of the log.

grep <term> webserver.log | awk -F\” ‘{print $2}’ | grep -c ‘<version>’

I have yet to see many CE/Vista URIs with the names of web browsers. So these are the most common versions one would likely find (what to grep – name – notes):

1. MSIE # – Microsoft Internet Explorer – I’ve seen 5 through 8 in the last few months.
2. Firefox # – Mozilla Firefox – I’ve seen 2 through 3.5. There is enough difference between 3 and 3.5 (also 2 and 2.5) I would count them separately.
3. Safari – Apple/WebKit – In searching for this one, I would add to the search a ‘grep -v Chrome’ or to eliminate Google Chrome user agents.
4. Chrome # – Google Chrome – Only versions 1 and 2.

Naturally there many, many others. It surprised me to see iPhone and Android on the list.

She pointed out students are the most affected by and most important constituent for any decisions we make. Yet the student point of view is almost never considered. Capturing what is good for students might mean installing all the possibilities where students and faculty could compare. It might mean surveys, however, I think self-reporting provides so much erroneous data we could do without it.

My job’s focus is more toward what is the most efficient, least problematic system for me to start/stop, install, upgrade, and review logs. I am still mulling what job position we have who would focus on ensuring whatever we do will provide for the best student experience. Guess really that should be all of us.

Course Instructor can produce a report of the raw tracking data. However, access to the role falls under the Administration tab so people running the system need to make a user specifically to enroll themselves at the course level to get the reports. (Annoying.)

Instead the administrators for my campuses pass up to my level of support requests to generate reports. For providing these I have SQL to produce a report. This example is for users who clicked on a specific file. Anything in bold is what the SQL composer will need to alter.

set lines 200 pages 9999
col user format a20
col action format a32
col pagename format a80

clear breaks computes
break on User skip 1
compute count of Action on User

select tp.user_name "User",ta.name "Action",
      to_char(tua.event_time,'MM/DD/RR HH24:MI:SS') "Time",
      NVL(tpg.name,'--') "PageName"
  from trk_person tp, trk_action ta, trk_user_action tua,
      trk_page tpg, learning_context lc
  where tp.id = tua.trk_person_id
    and ta.id = tua.trk_action_id
    and tua.trk_page_id = tpg.id (+)
    and tua.trk_learning_context_id = lc.id
    and lc.id = 1234567890
    and tpg.name like '%filename.doc%'
  order by tp.user_name,tua.event_time
/

Output

• User aka tp.user_name – This is the student’s account.
• Action aka ta.name – This is an artifact of the original script. You might drop it as meaningless from this report.
• Time aka tua.event_time – Day and time the action took place.
• PageName aka tpg.name – Confirmation of the file name. Keep if using like in a select on this.

Considerations

I use the learning context id (lc.id aka learning_context.id) because in my multi-institution environment, the same name of a section could be used in many places. This id ensures I data from multiple sections.

The tricky part is identifying the file name. HTML files generally will show up as the name of in the title tag (hope the instructor never updates it). Office documents generally will show as the file name. Here are a couple approaches to determining how to use tpg.name (aka trk_page.name).

1. Look at the file in the user interface.
2. Run the report without limiting results to any tpg.name. Identify out of the results the name you wish to search and use: tpg.name = ‘page name‘

Most tracked actions do have a page name. However, some actions do not. This SQL is designed to print a “–” in those cases.

Review and revise, as needed, policies, such as student absenteeism policies and sick leave policies for faculty and staff, that make it difficult for students, faculty, and staff to stay home when they are ill or to care for an ill family member. Do not require a doctor’s note to confirm illness or recovery.

Some faculty use online learning systems to provide notes to students so students can concentrate on the lecture or discussion rather than furious copy the blackboard or projection. The minority place all the information from lectures and slides online so students don’t get too far behind if they miss a class. If entire faculties did this for all the lectures, then students with the flu could stay home in isolation withouth much ill effect this flu season.

Of course, working to place all this material online is a ton of work to place on the shoulders of the faculty.

Also it would be an enormous weight on the shoulders of my colleagues who manage our storage.  I would find it unlikely we host more than 1/5 of their

Photo: My mother demonstrating how she spends time outside during fire season. Nothing to do with the flu except some people cover their mouth and nose to protect themselves.

Market share numbers annoy me. These are always self-reported numbers from a survey. The sample sizes are almost always not very impressive and when broken down doesn’t really represent the market. DI didn’t post a link to where they got the numbers just the name of the group. Some digging and turned up this Background Information About LMS Deployment from the 2008 Campus Computing Survey. For background information it is woefully lacking in important information such as sample size, especially the breakdown of the types of institutions in the categories.

The numbers DI quotes of CC are very different for the same year the Instructional Technology Council reports: Blackboard market share 66% (DI/CC) vs 77% (ITC). An 11% difference makes is huge when the next largest competitor is 10% (DI/CC).

Other missing critical information: Are these longitudinal numbers, aka the same respondants used participate in every year the survey quotes? Or is there a high turnover rate meaning an almost completely different set of people are answering every year so the survey completely relies on the randomness of who is willing to answer the survey? So the numbers could shift just because people refuse to answer giving Blackboard reduced market share only because Moodle customers are more willing to respond to questions about it?

Most of the major LMS products on the market started at a university or as part of a consortium involving universities. I knew the background of most of the products on in Figure 1. Somehow I never put that together.

Will another university take the lead and through innovation cause the next big shakeup? I would have thought the next logical step to address here in the DI presentation would be the innovative things universities are doing which could have an impact. Phil described Personal Learning Environments (not named) as potentially impacting the LMS market, but he was careful to say really PLEs are an unkown. The were no statements about brand new LMSs recently entering or about to enter the market.

Figure 1: Start year and origin of LMSes. Line thickness indicates market share based on Campus Computing numbers. From the DI WebEx.

When people use my project as an example, it gets my attention. GeorgiaVIEW was slightly incorrectly described on page 26 Trends: Changing definition of “centralization”.

1. We do not have an instance per institution which has a significantly higher licensing cost. We do give each institution their own URL to provide consistency for their users. Changing bookmarks, web pages, portals, etc everywhere a URL is listed is a nightmare. So we try to minimize the impact when we move them by a single unchanging URL.We have 10 instances for the 31 institutions (plus 8 intercampus programs like Georgia ONmyLINE) we host. Learn 9 will not have the Vista multiple institution capability, so should we migrate to Learn 9 an instance per institution would have to happen.
2. We have two primary data centers not have a primary and a backup data center. By having multiple sites, we keep our eggs in multiple baskets.

The primary point about splitting into multiple instances was correct. We performed the two splits because Vista 2 and 3 exhibited performance issues based on both the amount of usage and data. With ten instances we hit 20,000 4,500 users (active in the past 5 minutes recently) but should be capable of 50,000 based on the sizing documents. We also crossed 50 million hits and 30 million page views. We also grow by over a terabyte a term now. All these numbers are still accelerating (grows faster every year). I keep hoping to find we hit a plateau.

Figure 2: LMS consortia around the United States. From the DI WebEx.

All this growth in my mind means people in general find us useful. I would expect us to have fewer active users and less data growth should everyone hate us. Of course, the kids on Twitter think GeorgiaVIEW hates them. (Only when you cause a meltdown.)

UPDATE: Corrected the active users number. We have two measure active and total. 20,000 is the total or all sessions. 4,500 are active in the past 5 minutes. Thanks to Mark for reading and find the error!

Collisions on names are common enough. For example, here are a couple names our clients use to brand their sites which other places also use.

• uLearn – Georgia State University and University of Miami and University of Alaska Fairbanks
• elearning – Gainesville State College and Texas A&M University

My own project, GeorgiaVIEW is not immune. Some time ago I noticed the GeorgiaView Consortium (geological remote sensing) at the University of West Georgia.

I guess it is a good thing one Bbworld is in July and the other is in September.

For now I’ll just drop my RSS feed for the hashtag.

The one I see most often is comparing computers to a car. This morning I saw this on an email list describing a person’s experience  migrating to Vista 8 from Vista 3.

It is like I have traded in a familiar (though frustrating) car for one that has the lights, wipers, and radio in new locations.

Also this morning, Vista 8 was compared to a malfunctioning pen forced on faculty who would rather use a better pen. Nevermind all pens are not used exactly the same. (Fountain vs rollerball) Some require more maintenance and care than others.

A coworker always says Free Open Source Software like Sakai or Moodle are free as in free puppies not free beer. Nevermind proprietary bought systems like Blackboard are bought as in bought puppies.

weblogic.diagnostics.lifecycle.DiagnosticComponentLifecycleException: weblogic.store.PersistentStoreException: java.io.IOException: [Store:280036]Missing the file store file “WLS_DIAGNOSTICS000001.DAT” in the directory “$VISTAHOME/./servers/$NODENAME/data/store/diagnostics”

So I looked to see if the file was there. It wasn’t.

I tried touching a file at the right location and starting it. Another failed start with a new error:

There was an error while reading from the log file.

So I tried copying to WLS_DIAGNOSTICS000002.DAT to WLS_DIAGNOSTICS000001.DAT and starting again. This got me a successful startup. Examination of the WLS files revealed the the 0 and 1 files have updated time stamps while the 2 file hasn’t changed since the first occurance of the error.

That suggests to me Weblogic is unaware of the 2 file and only aware of the 0 and 1 files. Weird.

At least I tricked the software into running again.

Some interesting discussion about these files.

1. Apparently I could have just renamed the files. CONFIRMED
2. The files capture JDBC diagnostic data. Maybe I need to look at the JDBC pool settings. DONE (See comment below)
3. Apparently these files grow and add a new file when it reaches 2GB. Sounds to me like we should purge these files like we do logs. CONFIRMED
4. There was a bug in a similar version causing these to be on by default.

Guess that gives me some work for tomorrow.

• Scott Kodai
• Paty Savage
• Peggy Collins
• Laura Gekeler

Scott found the best point, I think.

Compliance doesn’t work to create value. Compliant work will always go to the lowest bidder. We can always find someone cheaper to follow the manual. Value is created by doing something different.

See! This is a mind numbingly positive message.

I liked some people on Twitter pointed to Jeff Longland’s role with VistaSWAT as a leader in the vacuum Blackboard has left open in the community.

Created a Yahoo Pipe for Bbworld09.

UPDATED 2009-07-15:

This TED video has much of the same substance as Godin’s Bbworld keynote.

Please share any feedback with me directly at a Web site created for just that purpose called Tell Michael Chasen.  I will personally read every message I get, so please let me know what you think.

That page in the email is no longer available.

I guess it was moved in the web site redesign for the launch for Learn 9 (Jan 27), then this page lasted at its original location all of 14 days. The new page is Tell the CEO.

A redirect to get those of us who refer to the emails we have received to the right page would be nice.