luminis

AMLS Rec Engine

Now anyone can tap into the Amazon Machine Learning Service (AMLS)?

I’d love to see Ellucian’s Banner product make smart recommendations based on AMLS. Students register for university classes through Banner. I could see Banner leveraging AMLS to figure out that students who like certain classes might be interested in other classes. There already is a market trying to improve retention by detecting students who have trouble on certain classes or tests or questions get flagged for extra attention. I could see AMLS helping the other spectrum. Students who take a certain class might be interested in certain cooperative education opportunities, clubs, or campus events. Or vice versa, such as students who join the environmental protection club might be interested in an elective class about the impact of biological pest controls vs pesticides.

Or maybe AMLS going beyond class recommendations is more appropriate for Ellucian’s Luminis portal product.

Higher education is full of opportunities for connect students to things about which they are oblivious. Flyers on various walls easily miss students. Enormous daily emails go straight to junk mailboxes. Students (and staff and faculty) deserve smarter ways to connect to the things that will make their experience better. Done right, I could see AMLS filling that need.

Preserving CE/Vista Settings

I’ve been asked for notes about this a few times. So here’s a blog post instead.
🙂

A coworker is working on scripting our updates. We lost the Luminis Message Adapter settings in applying the patch to the environment we provide to our clients. Fortunately, those settings are maintained by us not our clients. So I pushed those settings back very easily. Unfortunately, it points to the need to capture the settings for the potential purpose of restoring the settings.

In Oracle databases, this is pretty easy. As the schema user, run the following. It does some intentional things. First, we have multiple institutions, so the breaks make identifying which institution easier. Second, the same label for multiple forms gets confusing, so I am sorting by setting description id under the theory these ids are generated at the time the page is created, so the same tools will float together. (The last modified time stamp is probably unnecessary, I used it in an earlier version and left it just in case Vista for whatever reason added a new setting for the same label instead of modifying the existing one.) This can be spooled both before and after the upgrade. Use diff or WinMerge to compare the versions. Anything lost from the before version should be evaluated for inclusion adding back to the settings.

col lc_name format a50
col setting_value format a80
col label format a80
col lock format 999
col child format 999

clear breaks computes
break on lc_name skip 1

select learning_context.name lc_name, settings_description.label, settings.setting_value,
settings.locked_flag “lock”, settings_description.inheritable_flag “child”
from learning_context, settings, settings_description
where settings.settings_desc_id = settings_description.id
and settings.learning_context_id = learning_context.id
and learning_context.type_code in (‘Server’,’Domain’, ‘Institution’,’Campus’,’Group’)
order by learning_context.name, settings.settings_desc_id
/

An example of the multiple forms issue is external authentication. CE/Vista provides an LDAP (A) and an LDAP (B). The settings_description.label for both is contextmgt.settings.ldap.source. The settings_description.name for both is source. It looks like each of the two identical labels has a different settings.settings_desc_id value depending on whether it is A or B. To me it seems lame to use the same label for two different ids.

The most vulnerable parts of the application to lose settings during an update are the System Integration settings. A mismatched Jar on a node will wipe all the settings associated with that Jar.

However, I can see using this to capture the settings as a backup just in case an administrator or instructor wipes out settings by mistake. Yes, this is scope creep. Create a backup of the settings table to actually preserve the settings.

create table settings_backup_pre_sp2hf1 tablespace WEBCT_DATA as select * from settings;

Contexts: As a server admin, I maintain certain settings and push those down. Each client has control over some other settings and may push those down from the institution context. Maybe some are creating division and group admins? Maybe some instructors are changing things at the course or section levels. I may end up capturing everything?

Restoration: The whole purpose of preserving the settings is to restore them later. There are a couple methods in theory:

  1. Providing the settings to a human to re-enter. The labelling issue makes me question the sanity of trying to explain this to someone.
  2. Update the database directly would just need settings.id ensure it is the right location. Maybe dump out the settings in the format of an update command with labels on each to explain the context? Ugh.

If settings were not so easily lost, then this would be so much easier.

View: Another table of interest is the settings_v view. (Redundant?) The only reason I don’t like this view is it reports the values for every learning context which makes reporting off it much, much longer. For example, the encryption key for a powerlink is listed 8 places in settings/settings_description and 18,769 places in settings_v.

LC Oddities

IMS XML for Blackboard Vista 8:

Say Division1 exists. We want to create Group1 inside Division1. Ignore that Division1 already exists and write XML to create it again. Create Group1 with relationship tag info for Division1.

Starting with Group1 doesn’t work unless command-line overrides starting learning context to be Division instead of Group.

Luminis XML for Blackboard Vista 8:

Starting with Group1 fine because divisions are unsupported.

Don’t ever use Luminis XML as a model for IMS. Ever!

CE/Vista and Banner Integration

This is the second time I have worked on making Vista integration work with Banner. The first was 2005 in Vista 3.0.3 at Valdosta State. The production here at GeorgiaVIEW was set up by Harold, Jill, and Amy years ago and integrated into the install scripts or part of the cloned databases.

So now I am working on getting it to work in Vista 8. The IMS imports worked the first time like a charm. When I turned to using the Luminis adapter, the person records worked fine but the group contexts failed in Vista 8 and worked fine in Vista 3. So the “siapi.sh luminis import restrict” works fine.

Command-line

We have 41 institutions in Vista 3 currently. So imports are automated to some degree to preserve the sanity of Jill (and to a lesser degree Amy and myself). Rather than put in the UI all the settings, we have a properties file defining the location, glcid, sourcedid.source and sourcedid.id for each institution. This allows us to easily pass the values when importing at the command-line.

My first approach was to leave the settings identical to what I used to create persons and group records with IMS. This essentially uses the glcid of the institution and sourcedid of the institution. This is what resulted in the person records working and groups not. Fail.

I realized my error in logic must be the lack of a division-to-group relationship as the error described the groups cannot be related to an institution. So I changed the properties to use the division values for the sourcedid. Fail.

So I went looking in “Guide to Integration with the SunGard Luminis Data Integration Suite” for what I ought to use at the command-line. I didn’t find a solution. Just the same command-line lacking even the glcid and sourcedid.
🙁

XML

Giving up on the command-line approach for now, I added the relationship element to the XML so the group would become a child of one of the divisions I created with IMS. It sorta worked! The groups all imported but the course failed with the exact same error the groups formerly succeeded. To add insult to injury, simply running the import again on the exact same file had the courses import.

Mistakes

A mistake I made was reading the documentation: “Guide to Integration with the SunGard Luminis Data Integration Suite”.

Sungard Libraries:

  1. Page 8 says imq.jar and mbclient.jar do not come with CE/Vista and must be obtained from Sungard. All three of us thought in Vista 3.x these were automatically placed so we didn’t need to place them. Best I can tell, these were installed by Vista. I found $WEBCTDOMAIN/customconfig/startup.properties references both files in CUSTOM_CLASSPATH and setEnv.sh references CUSTOM_CLASSPATH. (This document has notes for what CE customers need to do and no note about CE users needing to go get them from Sungard.)
  2. Those who believe the last note would keep reading and find on Page 9 instructions to deposit the files in $WEBCTDOMAIN/serverlibs/. Assuming I am wrong about item #1, the startup.properties expects them in $WEBCTDOMAIN/serverlibs/luminis/ and would not find them where the document says to put them.

If a Learning Management System Was a Boat…

I saw a CompSci.ca blog post for If a programming language was a boat… (through WIRED) and laughed at the descriptions of C, ColdFusion, Java, Perl, PHP, and Ruby.

Java is a cargo ship. It’s very bulky. It’s very enterprise~y. Though it can also carry a lot of weight. Will carry a project, but not very fun to drive.

PHP is a bamboo raft. A series of hacks held together by string. Still keeps afloat though.

Blackboard Vista / CE 6+

Vista is an aircraft carrier. Bulky, enterprise-y, and a floating city which requires an entire armada of support (SafeAssign, Wimba, Luminis) to really fulfill its billed purpose. We see the “Mission Accomplished” banner with each new release. Marketing likes to describe it as a cruise liner, but I am sure it is because they are just recycling the Academic Suite points.

Blackboard Academic Suite

Academic Suite is a cruise liner. Swimming pools, mini-golf, and all you can eat buffets with little of substance.

Sakai

Sakai is Noah’s Ark. Good in the great flood and built by divine guidance. However, we live in the present reality… not ancient Babylonian mythology or the Jewish mythology which adopted the story. Do you really want to take a trip in something that may not even exist?

Moodle

Moodle is a houseboat. Yes, it is on the water, so technically it is a boat, but you only really use it tied up to a dock. You don’t want to take it out on the ocean or try to have too many people use it.

Anyone have thoughts on Desire2Learn, ANGEL, or any other systems?

Rock Eagle Wrap-Up

Index of posts:

  1. RE 2007: GeorgiaVIEW Meeting (Pre-Conference)
  2. RE 2007: Birds of Feather: GeorgiaVIEW Vista
  3. RE 2007: Top Ten Disruptive Trends
  4. RE 2007: Birds of a Feather: Luminis
  5. RE 2007: Administering Sakai
  6. RE 2007: GeorgiaVIEW Vista File and Content Sharing
  7. RE 2007: USG Digital Content Repositories: Resources to Share

After this point, I got wrapped up in other things, moderating, fireworks, a Texas Hold ‘Em tournament, and dealing with tickets. The above are all sessions which affect my area even tangentially. Hope you enjoy.

RE 2007: Birds of a Feather: Luminis

Kathy Kral, University of West Georgia

  • Luminis IV CPIP connector: Clayton, Augusta, West GA.
  • Issues with IMS when allowing IMAP from outside the Luminis application? VSU no issues with IMAP and POP allowed outside. Augusta initially restricted but opened up access without advertisement. Mention on one of email lists Blackberries fail to do a close with connection resulting in stale sessions accumulating.
  • Mailbox sizes: West GA 10MB, Augusta 40MB for students and 125MB for faculty, Valdosta 250MB.
  • Makarand Kulkarani, Sungard – Created a replacement so mailbox is Gmail. This is a professional service engagement. Engagements start at 90 hours with unused hours un-billed. When GCF connectors are built Training similar, typically 48 hours.
  • Name changes are a pita.

Jesse Lyman built a Luminis to Vista 4 single-sign on connector to handle multiple institutions.

BbWorld Presentation Redux Part I – Automation

Much of what I might write in these posts about Vista is knowledge accumulated from the efforts of my coworkers.

I’ve decided to do a series of blog posts on our presentation at BbWorld ’07, on the behalf of the Georgia VIEW project, Maintaining Large Vista Installations (2MB PPT). I wrote the bit about tracking files a while back in large part because of the blank looks we got when I mentioned in our presentation at BbWorld these files exist. For many unanticipated reasons, these may not be made part of the tracking data in the database.

Automation in this context essentially is the scheduling of tasks to run without a human needing to intercede. Humans should spend time on analysis not typing commands into a shell.

Rolling Restarts

This is our internal name for restarting a subset (consisting of nodes) of our clusters. The idea is to restart all managed nodes except the JMS node, usually one at a time. Such restarts are conducted for one of two reasons: 1) have the node pick up a setting or 2) have Java discard from memory everything. The latter is why we restart the nodes once weekly.

Like many, I was skeptical of the value of restarting the nodes in the cluster once weekly. Until, as part of the Daylight Savings Time patching, we provided our nodes to our Systems folks (hardware and operating systems) and forgot to re-enable the Rolling Restarts for one batch. Those nodes starting complaining about issues into the second week. Putting back into place the Rolling Restarts eliminated the issues. So… Now I am a believer!

One of my coworkers created a script which 1) detects whether or not Vista is running on the node, 2) only if Vista is running does it shut down the node, 3) once down, it starts up the node, and 4) finally checks that it is running. Its pretty basic.

Log cleanup to preserve space

We operate on a relatively small space budget. Accumulating logs infinitum strikes us as unnecessary. So, we keep a months’ worth of logs for certain ones. Others are rolled by Log4j to keep a certain number. Certain activities can mean only a day’s worth are kept, so we have on occasion increased the number kept for diagnostics. Log4j is so easy and painless.

We use Unix’s find with mtime to look for files 30 days old with specific file names. We delete the ones which match the pattern.

UPDATE 2007-SEP-18: The axis files in /var/tmp will go on this list, but we will delete any more than a day old.

Error reporting application, tracking, vulnerabilities

Any problems we have encountered, we expect to encounter again at some point. We send ourselves reports to stay on top of potentially escalating issues. Specifically, we monitor for the unmarshalled exception for WebLogic, that tracking files failed to upload, and we used to collect instances of a known vulnerability in Vista. Now that its been patched, we are not looking for it anymore.

Thread dumps

Blackboard at some point will ask for thread dumps at the time the error occurred. Replicating a severe issue strikes us as bad for our users. We have the thread dumps running every 5 minutes and can collect them to provide Blackboard on demand. No messing with the users for us.

Sync admin node with backup

We use rsync to keep a spare admin node in sync with the admin node for each production cluster. Should the admin node fail, we have a hot spare.

LDIS batch integration

Because we do not run a single cluster per school and the Luminis Data Integration Suite does not work with multiple schools for Vista 3 (rumor is Utah has it working for Vista 4), we have to import our Banner data in batches. The schools we host send the files, our expert reviews the files and puts them in place. A script finds the files and uploads each in turn. Our expert can sleep at night.

Very soon, we will automate the running of the table analysis.

Anyone have ideas on what we should automate?