webct-users@anu.edu.au
[Top] [All Lists]

[Webct-users] Re: WebCT usage 'measures'

Subject: [Webct-users] Re: WebCT usage 'measures'
From: Jenny Edwards
Date: Fri, 16 May 2008 16:30:23 +1000
On 13/5/08 5:49 pm, "James Meek" <james.meek@xxxxxxxxxx> wrote (in part):

> ... [a recent arrival from Deakin] was wondering aloud if we knew how many/how
> much/at what ?level¹ people were doing ?online¹ in her Faculty. ...
>  
> At Deakin they were pretty regimented and monitored, by the sounds.  You know:
> ?level 1¹ (required: administrivia/basic communications), ?level 2¹
> (additional small interactive stuff), ?level 3¹ (?fully online¹, more
> sophisticated functions/content)?, or somesuch.
>  
> ... Do we have any ?metrics¹ re numbers of ug/pg course sites, and number of
> these that are actually active, within the boundaries of [Faculty]? And beyond
> this, have you made any effort to determine/measure the ?depth¹ (and varying
> ?attributes¹) of the ?active¹ sites?

G¹day James

Thanks for a great question, and I hope you don¹t mind my sending this
answer to a wider group.

The short answer:
Look at WebCT > ANU Designer's Resource Site > ToolUse Reports > 2008 - all
reports > [pick your faculty or subject etc] > Summary (lower left) >
courses by requests/user

The long answer:
We have a handful of ways we can generate stats about WebCT use, but
interpreting the data into meaningful measures is far from simple.

We have had a similar series of 5 levels defined.  Our levels 3-5 correspond
approximately to the levels 1-3 you've described.  Ours came largely through
work done by Chris Trevitt, culminating in a recommendation that all units
have a web site meeting the requirements of the first 3 levels ­ see
attached.  This was the result of discussions through 1999, with a view to
implementation through 2000-2003.

One upshot of this was the decision to automatically create WebCT sites for
every course in the system (minus the research courses) -- ie sites for over
4000 courses each year.  These are based on a template populated with as
many of the items in those first 3 levels that we could figure out a way to
automatically include: a link to the handbook description, the course¹s
timetable, a link to the Library reserve list for the course; plus a few
generic items such as help resources for students; and some template items
to make it as easy as possible for lecturers to publish their name, e-mail
address, office hours etc.  Some areas have modified the basic template to
create something more specific for their subject.  As a result, all our
sites are at least are at least about level 1.5 on Chris' scale.

3-4 years ago a couple of areas carried out a comprehensive audit of their
sites to evaluate the "depth" of their resources/development.  Some areas
have one person (eg a departmental administrator) who checks every site and
makes sure certain information is included, generally aiming around Chris'
level 3.  However, from a central point of view, manually accessing and
assessing each of 4000+ sites is not something I'm keen to take on!  A few
times we've discussed implementing some sort of campus-wide quality
assurance, but the conclusion so far has been that it has to be left to the
areas.

We have a number of different ways we try to analyse WebCT usage.  The most
appropriate for your question is a thing, called ³ToolUse², which counts how
many times each WebCT tool (eg discussion board, assignment tool, home page,
etc) is used.  This analysis can be run for individual courses, on request.
We also run it automatically every week for various groups of courses.  The
groups are: undergraduate, post graduate and ³other²; each subject code (eg
ATRH, CHEM, JPNS etc); and the subject codes aggregated for their faculties
(although these lists are due an update).  These reports can be accessed
through WebCT > ANU Designers¹ Resource Site > ToolUse reports.

Once you find the report you want, ToolUse is mostly interested in showing
you patterns of tool-use throughout the semester, eg you might discover that
the discussion board activity peaked in week 4.  However, in the lower-right
corner of the index page for each report, you'll find a drop-down list of
"summaries", including "Courses by requests/user".  This is the simplest,
most direct measure we have of site activity.  You can download this data
using "courses, tab delimited data" also from the summary menu.

Good luck interpreting the data!  I've tried a few times to generalise, with
mixed success.  I think the last time we tried this we decided that 30+ was
likely to be level 3 compliant, 100+ was "active" and 300+ was "very
active", but you need to look at a few sites, especially around your
cut-points to see what you think -- eg I've seen sites with 40+ hits per
user that are untouched by the lecturer (hopefully implies that our template
elements are useful!), and sites around 20 hits per user that are perfectly
acceptable, neat little administrative sites.

Some other metrics you might like to look at:
 - "track pages" and "track students" within specific sites give you details
of individual site usage
 - http://webct.anu.edu.au/logs -- "webalizer" analysis for the whole server
 - http://webct.anu.edu.au/usage -- some interesting views on usage of the
whole server

Hope this helps!

Jenny
-- 
Jenny Edwards
Online Environments Support Group
Information Support Services Program, Division of Information
mailto:webcthelp@xxxxxxxxx
phone: 612 58886

_______________________________________________
Webct-users mailing list
To send messages to the list, email to Webct-users@xxxxxxxxxx
To unsubscribe , view archived messages or edit list options visit 
http://mailman.anu.edu.au/mailman/listinfo/webct-users
<Prev in Thread] Current Thread [Next in Thread>
  • [Webct-users] Re: WebCT usage 'measures', Jenny Edwards <=