Announcing QTI Works

Development work during the first few months of the QTIDI project has been progressing successfully, though you would certainly be forgiven for doubting this! As well as me being notoriously bad at finding time to write blog posts, the first development iterations on QTIDI have focused on "refactoring" all of the QTI-related software components we have been building and using on recent JISC projects so that they make a much better foundation for the technical goals of this project. (Rubbish analogy time: I've taken lots of random things, laid them on a carpet, and hit them with hammers until they break into little pieces. I'm now joining these pieces up to make something nice. Hopefully. The analogy is rubbish as I can't think of any examples where I've made something nice out of random bits on a carpet. Luckily, I'm better at doing this with software, so probably shouldn't have used this analogy in the first place.)

One of the key bits of work done so far has been a redesign of JQTI, which is the Java software library for "doing" QTI stuff that Southampton developed a few years ago. I've previously blogged about why I thought this is necessary (see http://davemckain.blogspot.co.uk/2011/06/refactoring-jqti-jqti.html) and the result of this is coming together under the not-very-original name of JQTI+. This refactoring work is now almost complete, with the exception of QTI tests, which JQTI never quite implemented fully and will be revisited in a few months.

On top of JQTI+, I'm building the replacement for MathAssessEngine that will become the main technical deliverable of this project. MathAssessEngine, which was based on the original QTIEngine, is also going to be torn apart and redesigned so that it can do all of the things it now needs to do, and do them all really well.
To reflect the scope of the work we're doing, we've decided to give the replacement for MathAssessEngine a completely new name and, after a couple of months of riotously bad naming attempts, we've decided to call it QTI Works.

I will deploy regular public development snapshots of QTI Works while it takes shape over the next few months, which you will be able to find at:

http://www2.ph.ed.ac.uk/qtiworks

If you remember to wear a hard hat, you can go in and try the first development snapshot now. This showcases the brand new (and vastly improved) QTI 2.1 validation functionality in JQTI+, as well as demonstrating the newly-refactored rendering and delivery of QTI 2.1 assessment items. (You'll have to make do with a selection of sample items for the time being... the functionality that will allow you to upload, validate and try out your own items is still written on the back of some envelopes. This will turn into real code that you can play with during the next development iteration. Hopefully!)

Demos at the CETIS Conference

QTIDI and its sister project Uniqurate were represented at the CETIS Conference in Nottingham in February 2012. We demonstrated

• The Learning Tools Interoperability (LTI) Connector running a Common Cartridge test;


• The Uniqurate question authoring tool, which also has facilities for creating a contant package containing a question and its associated media and/or stylesheets;


• Interoperability between our tools and those of colleagues from Germany, France and Korea as well as the UK, using the latest version of MathAssessEngine, which is currently morphing into QTI Works.

QTIDI - Connecting QTI Assessment to VLEs

The QTIDI project, funded by JISC under the Assessment and Feedback programme, is preparing a package of software and documentation for transferring the QTI Works rendering and responding engine - a direct descendant of MathAssessEngine - and software to link it to popular VLEs to institutions in the HE and FE sector. The project will provide a documented and packaged version of the tools for distribution to Kingston University, Harper Adams University College and the University of Strathclyde. Should other institutions wish to join in the project as receiving partners at a later date, they will be made welcome. Alongside, we have the Uniqurate project, which is building on the Aqurate and Mathqurate editors to create a more intuitive, user-friendly authoring too for QTIv2.1 questions. Eventually it will also incorporate the functionality of Spectatus to provide a one-stop editor for questions and tests.

Here's a quick roundup from Niall and David about what’s going on in QTIDI:

Niall: “I have been working on getting a good IMS LTI example working in Java. (The IMS example software is written in PHP.) A basic LTI 1.0 test with consumer (LMS) and tool components is almost complete. I'll be writing some developer documentation for LTI containing this code as well as C# and PHP example code. I have also (with help from Dave) set up a MathAssessEngine NetBeans project on my Laptop. I have also been taking part in the IMS QTI working group meetings.”

David: “Most of the work so far has been continued refactoring of JQTI to make it more sensible, and an initial webapp framework for SonOf(QTI|MA)Engine. So, all this first iteration is going to do is provide a "validation service". This will let you upload a standalone item XML or item/test Content Package, which will then be fully validated and will generate a summary report. Once that's in place, next iteration will be refactoring the rendering of standalone items, as well as a first cut of the REST API for that, which will be the way in for LTI.”

Note that the original MathAssessEngine will be replaced during this project, so the appearance and the way things get uploaded will change. The test mode is needing improvements, so there will be some changes there too. This means, of course, that feedback from clients will be more useful at the moment in pointing out "would like to see" suggestions (and the reasons for them), which can then be matched up with the specification to make sure things work the way they should, rather than as a debugging exercise.

Updates on Spectatus

Spectatus has been updated to work with the new IMS Question and Test Interoperability (QTI) version 2.1 schema, and has been given some new functionality. Submission mode for a test can be set to either individual or simultaneous, and navigation mode may also be controlled.

Question handling has been improved: any errors encountered on import into a test are displayed in a new message area at the bottom of the Spectatus interface (so there is no longer any need to count your questions to check that an import worked), and it is now possible to preview a single question. The issue with trying to edit a question in Windows 7 has been resolved.

Filling in of metadata is now easier for test authors, in that many of the fields (such as author and institution) default to their values from the previous editing session. The main taxonomy has been standardised, while a user-defined secondary one may be specified. We would be interested to learn what additional metadata test authors would like to be able to maintain.

A download link and project information can be found here.

eAssessment Scotland Poster

The QTI IPS project was represented at eAssessment Scotland (Aug 26th/27th in Dundee).  I tweeted a photo when the poster was put up but only included the top half - here is the whole picture...

You can click this picture to enlarge it.

Dissemination and Wider Collection of Profile Examples

QTI Roadshows

We had a very busy few weeks of dissemination last month, with the Roadshows at Strathclyde and Southampton, followed by the CAA2011 Conference at which several members of the team were presenting. We were able to demo the current versions of the tools and show some more sophisticated question types used in a number of different contexts.

Profiling

As part of the profiling work, we are collecting examples of questions and tests to create a profile of users’ requirements. Although we have within the team a number of lecturers and tutors who use these resources with classes, we are conscious that there are many more disciplines out there that may have new and interesting variations on the kinds of questions that may be set. To get the widest coverage we can, we are asking you as users (and potential users) of e-assessment to contribute to our collection any questions, tests, or descriptions of these, which you as would like to be able to use. The intention is to explore the way in which these relate to the QTIv2.1 specification and extend the profile to include as many of these requirements as possible.

Just a reminder that the QTIv2.1 specification provides a variety of interactions which produce input that may be textual, numerical, graphical or selection type. Flexible feedback can be provided, targeted at the individual student’s response. Questions and tests can have several interrelated parts, and input from one part can be carried over into later parts of the question and may affect the behaviour of both the question and the test.

Examples are welcome from across education, but we are concentrating on the college and university environment. They can relate to any discipline or topic area, and may bring together several of these.

If you have examples in QTI format, they would be very welcome additions to the collection; we can convert resources submitted as QTIv1.2 to QTIv2.1 (using facilities openly available at http://code.google.com/p/qtimigration/ . However, if your examples are in other formats, or in the form of descriptive documents, please contribute them too, and we will investigate how they could be implemented in QTI.

Please contact Sue Milne (email: sue.milne@e-learning-services.org.uk) for further information, or to contribute your examples. Many Thanks!

Update on Profiling and Support

Issue Chasing and Profile

The technical members of the QTI-IPS team have been examining the specification and documentation in minute detail, attempting to tease out any anomalies which could get in the way of interoperability. We are finding that we need to separate the issues which are strictly QTI from those that are implementation-oriented. Sometimes this is quite tricky, as the representation of intentions in a question is not easily separated from its rendered form.

Updated versions of some of the renderers are now on the support site virtual machine, a copy of which is visible at http://cloud.niallbarr.me.uk/ - see later in this post.

We’ve just had a very positive meeting in Paris with the IMS QTI Working Group. We discussed many things about the specification and documentation and a list of elements for the Main Profile was suggested. Lively discussion revealed that there are many interpretations of the term “profile”, depending on who is talking! There are, in fact a number of profiles under construction, so the requirement for more than one profile to be described, implemented by more than one set of tools and demonstrated is sure to be fulfilled.

Support

QTI Roadshows

The QTI-IPS project is now moving into the support stage of its activity, beginning with yesterday’s Roadshow at Strathclyde University. We had a full day – a very busy day – in which we

• Looked at the reasons for using standards for educational resources, particularly questions and tests,


• Showed some sample questions and tests which take e-assessment a good bit further than the normal MCQ-MAQ-text input,


• Demoed lots of tools for rendering, editing and converting questions and tests, and had participants trying them out in a long and lively hands-on session,


• Discussed the issues around QTI and uptake

Altogether, it was a very useful meeting, certainly from our point of view, and the feedback from delegates was generally very positive, but coloured by the financial constraints within HE and FE.

The second of these Roadshows will be the pre-conference workshop at the CAA conference in Southampton on July 4th, see http://jisc.cetis.ac.uk/events/.

Website

The website is taking shape in the virtual machine, and is visible at http://cloud.niallbarr.me.uk/.

Currently it has

• Two of the renderers – MathAssessEngine and JAssess – available for trial runs of questions,


• A Moodle instance which will soon have a course with sample tests for different disciplines,


• A link to the Eqiat question editor, developed at the University of Southampton under the EASiHE project



Coming soon





• SToMP II, a QTI 2.1 implementation written in PHP,


• QTItalk interactive QTI 2.1 documentation,


• QTICat which can take a set of questions and report on which renderers are able to run them,


• Links to more editors, with a bit of info about what they do best,


• Links to conversion tools – these will convert batches of questions at the same time,


• A growing library of examples of items and tests, demonstrating the specification - we still have to annotate these...


• The FAQ collection – currently under construction,


• Repositories – Minibix





Mailing Lists and Forums




The best places to ask questions about QTI are the mailing list at ims-qti@lists.ucles.org.uk and on the IMS Forum. Members of the team are keeping an eye on these lists and taking an active part in responding to queries.