QTI version 2.1 final released.

Version 2.1 of the IMS Question and Test Interoperability (QTI) specification was released on 25 January 2003. It's taken a long time to get QTI 2.1 completed, however now that is has finally released hopefully there will be a significant uptake. It has been a long process, with a very large number of people involved over the years. QTI 2.1 is really completion of QTI 2.0, so the process of developing it has been continuous since the start of the 2.0 working group several years ago. QTI 2.1 adds support for tests, as well as enhancing the support for individual question types.

Further work will be needed to develop a range of profiles that specify subsets of the full specification that fit the requirements of specific groups of users. An entry-level profile already exists which duplicates functionality of the Common Cartridge profile of QTI 1.2, but is much simpler than the older profile as benefits from QTI 2's response processing templates which removes the need to individually specify duplicate response processing rules in each question where a single marking algorithm is to be used. This entry-level profile should be easy to import into almost all existing electronic assessment systems, however it is a very small subset of QTI 2.1, and does not provide any functionality that is not already widely available. Further profiles that address the needs of higher education are still needed.

As we came towards the end of the development process, we also realised that there were many things that we would have liked to include given unlimited time and resources. Hopefully we will be able to develop some of these ideas as extensions which can later be merged into a future iteration of the specification. One which I would like to see is the ability to define graphs generated from templates, and taking input in the form of lines or points being added. In the time since we started QTI 2.0 web technology has moved on, and with HTML 5 this type of interaction should be very feasible. I am glad however that we made many of the decisions about the specification before HTML 5 arrived, because the XHTML base provides all of the flexibility needed, whilst being simple enough that it does not create a great obstacle to the specification being used with delivery systems other than a web browser.

Niall

LTIQuizzes

Quite early on in the QTIDI project I decided to write a demonstration application that would allow us to test LTI concepts along with a quiz system. The result was LTIQuizzes, a very simple quiz application that allows entry-level QTI 2.1 quizzes to be played through an LTI connection from a VLE. The I should really have blogged about this ages ago, however summer is a very busy time for me with conferences and software upgrades and feature development for the new academic year.

LTIQuizzes consists of an LTI connector (which is intended to be reusable software that can be linked to other systems), my original QTI 2.0 item player, APIS, and a new very basic QTI 2.1 assessment player which will eventually become part of my desktop Common Cartridge viewer, CCplayr. LTIQuizzes served a useful purpose for the QTIDI project, because it allowed us to demonstrate how QTI assessments can be linked into VLEs using LTI before our proper QTI delivery system, QTIWorks, was LTI enabled.

From the teacher's viewpoint LTIQuizzes is very straightforward to use. The teacher creates a new resource in the VLE, and clicks on the resource link which takes them into LTIQuizzes. As they are a teacher in the VLE course they are given the LTI "instructor" role, and so are shown the teacher screen in LTIQuizzes. This allows them to upload a packaged QTI question or test which will be displayed to any students that click on the VLE link into LTIQuizzes. Provided all the questions the quiz are automatically marked (i.e. there are no essay or extended test questions) LTIQuizzes will return a score to the VLE if it supports LTI version 1.1 when the student completes the assessment.

LTIQuizzes is intentionally extremely simple, and there is no way to create an activity without using an LTI link. David has taken a slightly different approach with the LTI connection in QTIWorks, where teachers set up the quiz after logging into QTIWorks directly, and then get the LTI information to configure an activity in Moodle, Blackboard or any other suitable VLE. This approach works very well with the LTI implementations that are now distributed as part of Blackboard 9 and Moodle 2.3, but is awkward with the Moodle 1.9 LTI plug-in, which assumes that administrators rather than teachers will put in most of the LTI information. When I was writing LTIQuizzes s the only LTI enabled VLE that I had access to was Moodle 1.9 with the LTI plug-in wa, so naturally I used an approach which fitted very closely with the approach taken by the developers of that version of Moodle plug-in.

Now the QTIWorks is fully LTI enabled, LTIQuizzes is much less important for our project, however I do intend to do some further work on it as it is sometimes useful to have a very simple system available for experimenting with. I will be doing some refactoring of to the APIS item player over the next few months to make it use generics properly (it's very old), and will probably use it for any experiments I do for ideas that might go into an eventual QTI 2.2. I will also be integrating the item and test player parts of LTIQuizzes into QTIPlayr so that it supports a variant of common cartridge that uses QTI 2.1 entry level assessments. The LTI interface is part of an ongoing project to provide simple lightweight LTI code in Java, C# and PHP that tool providers can use to quickly add LTI support to their software.

Enhanced item rendering in QTIWorks

After another flurry of activity, I've just finished and released the 8th development snapshot of QTIWorks for folk to play around with:

https://www2.ph.ed.ac.uk/qtiworks

This iteration of work has focused mainly on the delivery of single assessment items to candidates, joining up various bits of pipework that control this process and adding in some new rendering features and options that we'll need later for test rendering (and are quite useful in their own right too).

I'm afraid you'll still have to make do with playing around with the sample items that have already been loaded into the system for the time being. However, you'll probably be pleased to hear that the next work iteration will add in the first round of functionality for getting your own assessments into the system and doing fun things with them. (At last!) To make the wait a bit easier, I have gone through the existing samples and selected a reasonable set of options for how they should be delivered, which makes them much more fun to play with, as well as more illuminating to people who are less familiar with QTI.

Key changes in this release (1.0-DEV8)

• Assessment item rendering now incorporates the following new features:

1. Display of a model solution (via the QTI <correctResponse>).
2. The ability to reset a session back to the state it was in immediately after the last run of template processing, which effectively clears all candidate input back to the original state, but leaves randomly-chosen things intact. Existing "reinit" functionality has been made a bit clearer.
3. An explicit "closed" state, which is either entered explicitly by the candidate, or when an adaptive item becomes complete, or when the number of attempts hits your chosen value of maxAttempts. I think this will need a bit more work, as it was never implemented in QTIEngine or MathAssessEngine.
4. A new "playback" feature lets the candidate step through every interaction they made with the item, which is a possible way of implementing the allowReview concept from test rendering.

• A number of "knobs" are now available for controlling how you want to render a single item, including:

• maxAttempts (as seen in tests)
• author mode on/off (controls whether authoring hints are shown)
• a simple prompt to be shown at the top of the question (similar to rubrics in tests, but simpler)
• restrictions on what the candidate can do, including:
• close a session explicitly when interacting
• play a session back when closed
• reinitialize a session when interacting or when closed
• reset a session when interacting or when closed
• see a model solution when interacting or when closed
• see the result XML
• see the item XML source
• I have gone through all of the existing samples and set the above "knobs" to values that show each in their best light. For example, some suit a very formative "try as many times as you like" approach with rich feedback, others are more austere so are better displayed in a more rigid way. This should make the samples much more useful for people trying them out.
• The authoring debugging information has been improved, and now shows bad responses (e.g. string submitted to a float), and invalid responses (e.g. wrong number of choices made).
• There's now a rich database structure underpinning all of this, which records everything the candidate does and the changing QTI state during this process. This is currently used to implement the "playback" functionality, and will prove invaluable for analysing result data when the system delivers real tests.
• The HTTP calls that control the typical delivery of a "candidate session" via a browser are now as RESTful as you would pragmatically expect in this type of scenario. A more formal RESTful web service API will be trivial to do from this, but I'm going to hold off until anyone actually needs it.
• HTTP responses have been improved so that they always include the Content-Length headers. Responses that don't change also send cache-friendly HTTP headers such as ETag and Cache-Control.

Short term development roadmap

• Next up is adding in the web-facing functionality for creating your own assignments into the system, uploading content into them, validating them and trying them out (using the various "knobs" I listed above).
• After that it's finally time to implement the entry and exit points for candidates who get sent to QTIWorks to do an assessment. This is where LTI will come in.
• Then... tests. At last!

And relax.

Showcasing QTI 2.1 in QTI Works

As well as being a tool for managing, trying and delivering your own QTI assessments, QTI Works will also act as a showcase for QTI and the things it can do, much as the existing QTIEngine and MathAssessEngine already do.

With this in mind, I have started to feed Graham Smith's excellent collection of QTI examples into QTI Works, and plan to continue this process over the next few months while it takes shape.

These examples will be bundled into development snapshots of QTI Works for you to play around. So far, I have assimilated 3 sets of examples:

• The IMS standard examples (and a few extra bits)
• Examples demonstrating the MathAssess QTI extensions
• A small set of example items from language testing (German & Russian)

You can play around with these at: https://www2.ph.ed.ac.uk/qtiworks

(Yes, it does look very spartan at the moment, but don't worry about that!)

This exercise is actually very useful for this project for a number of other reasons:

1. It provides me with some sample data for running automated "integration tests" on the JQTI+ / QTI Works software. (For example, I feed them all of the code performing the various bits of QTI logic, such as reading in XML, validating the data models, writing out XML, running template processing, running response processing etc. This is invaluable for finding and fixing bugs, and making sure we can handle "real world" examples properly.) 
2. As well as being useful for integration testing, they also help with so-called "regression testing", which helps make sure that I don't break things inadvertently during the development process.
3. These examples have been around for a few years now, so this process is a good way of doing some QA on them and making sure they're right up to date with the QTI specification.

Enjoy!

Updates on Spectatus

Spectatus has been updated to work with the new IMS Question and Test Interoperability (QTI) version 2.1 schema, and has been given some new functionality. Submission mode for a test can be set to either individual or simultaneous, and navigation mode may also be controlled.

Question handling has been improved: any errors encountered on import into a test are displayed in a new message area at the bottom of the Spectatus interface (so there is no longer any need to count your questions to check that an import worked), and it is now possible to preview a single question. The issue with trying to edit a question in Windows 7 has been resolved.

Filling in of metadata is now easier for test authors, in that many of the fields (such as author and institution) default to their values from the previous editing session. The main taxonomy has been standardised, while a user-defined secondary one may be specified. We would be interested to learn what additional metadata test authors would like to be able to maintain.

A download link and project information can be found here.