I'm on my way out the door, but wanted to post this question. . .
As we're ending the Work Literacy course this week (don't worry--all the resources will be online for the forseeable future), the issue of defining "success" in an open, social media-enhanced course like this is definitely on my mind.
Should we define it by the number of people actively posting to the site? As with the CCK08 course, we've also seen a drop in activity in the forums and blog postings, but since we specifically gave people the opportunity to participate only as spectators, it's hard to know if a lack of forum postings should be our measure of success--or failure. I will say that when you see fewer discussions happening, there's a tendency to think that something's happening, but what that is, I can't say for certain.
Should we be looking at how many people visit the site? According to our Analytics, the visits each week to the site have been fairly steady, peaking early in the week and then falling down as we approach the weekend. This we expected, because we post each week's activities on Monday. There is also the option of following forum discussions via RSS, so we don't know how many people are participating that way.
Should "success" be defined by individual learners? That is, since this was an open course that allowed people to set their own levels of participation, should the measure of success really be a matter of individual opinion?
And none of this is getting at deeper levels of evaluation, like did people learn new skills and are they applying them to their personal work and their work with learners? We have some anecdotal evidence from participants that people were learning and trying out new things, but this was a very informal course, so what "proof" we have is pretty informal too.
I guess my big question is, how do we know if this was something we should do again? What should we be thinking about and how should we be measuring what we accomplished over the past several weeks?
I'd love to hear your thoughts on this, either in comments here or, if you're a member of the Work Literacy course, in the forum I set up to discuss the issue.