Progress testing and educational data mining

I read George Siemens's interview on O'Reilly Radar, How data and analytics can improve education, and was struck by an example he gave on how education could be changed with the use of analytics.

"In terms of evaluation of learners, assessment should be in-process, not at the conclusion of a course in the form of an exam or a test. Let's say we develop semantically-defined learning materials and ways to automatically compare learner-produced artifacts (in discussions, texts, papers) to the knowledge structure of a field. Our knowledge profile could then reflect how we compare to the knowledge architecture of a domain — i.e. "you are 64% on your way to being a psychologist" or "you are 38% on your way to being a statistician." Basically, evaluation should be done based on a complete profile of an individual, not only the individual in relation to a narrowly defined subject area."

An example of this in medicine is the formative assessment work, progress testing, developed in the Netherlands called Progress Test Feedback System PRoF. It is used at the Peninsula Medical School in the UK. Progress Test (Word Doc). OK it doesn't go as far as saying you are 45% doctor but it does allow students and their tutors to understand where they are on their learning trajectory.