Updates on Spectatus

Spectatus has been updated to work with the new IMS Question and Test Interoperability (QTI) version 2.1 schema, and has been given some new functionality. Submission mode for a test can be set to either individual or simultaneous, and navigation mode may also be controlled.

Question handling has been improved: any errors encountered on import into a test are displayed in a new message area at the bottom of the Spectatus interface (so there is no longer any need to count your questions to check that an import worked), and it is now possible to preview a single question. The issue with trying to edit a question in Windows 7 has been resolved.

Filling in of metadata is now easier for test authors, in that many of the fields (such as author and institution) default to their values from the previous editing session. The main taxonomy has been standardised, while a user-defined secondary one may be specified. We would be interested to learn what additional metadata test authors would like to be able to maintain.

A download link and project information can be found here.

eAssessment Scotland Poster

The QTI IPS project was represented at eAssessment Scotland (Aug 26th/27th in Dundee).  I tweeted a photo when the poster was put up but only included the top half - here is the whole picture...

You can click this picture to enlarge it.

Dissemination and Wider Collection of Profile Examples

QTI Roadshows

We had a very busy few weeks of dissemination last month, with the Roadshows at Strathclyde and Southampton, followed by the CAA2011 Conference at which several members of the team were presenting. We were able to demo the current versions of the tools and show some more sophisticated question types used in a number of different contexts.

Profiling

As part of the profiling work, we are collecting examples of questions and tests to create a profile of users’ requirements. Although we have within the team a number of lecturers and tutors who use these resources with classes, we are conscious that there are many more disciplines out there that may have new and interesting variations on the kinds of questions that may be set. To get the widest coverage we can, we are asking you as users (and potential users) of e-assessment to contribute to our collection any questions, tests, or descriptions of these, which you as would like to be able to use. The intention is to explore the way in which these relate to the QTIv2.1 specification and extend the profile to include as many of these requirements as possible.

Just a reminder that the QTIv2.1 specification provides a variety of interactions which produce input that may be textual, numerical, graphical or selection type. Flexible feedback can be provided, targeted at the individual student’s response. Questions and tests can have several interrelated parts, and input from one part can be carried over into later parts of the question and may affect the behaviour of both the question and the test.

Examples are welcome from across education, but we are concentrating on the college and university environment. They can relate to any discipline or topic area, and may bring together several of these.

If you have examples in QTI format, they would be very welcome additions to the collection; we can convert resources submitted as QTIv1.2 to QTIv2.1 (using facilities openly available at http://code.google.com/p/qtimigration/ . However, if your examples are in other formats, or in the form of descriptive documents, please contribute them too, and we will investigate how they could be implemented in QTI.

Please contact Sue Milne (email: sue.milne@e-learning-services.org.uk) for further information, or to contribute your examples. Many Thanks!

Update on Profiling and Support

Issue Chasing and Profile

The technical members of the QTI-IPS team have been examining the specification and documentation in minute detail, attempting to tease out any anomalies which could get in the way of interoperability. We are finding that we need to separate the issues which are strictly QTI from those that are implementation-oriented. Sometimes this is quite tricky, as the representation of intentions in a question is not easily separated from its rendered form.

Updated versions of some of the renderers are now on the support site virtual machine, a copy of which is visible at http://cloud.niallbarr.me.uk/ - see later in this post.

We’ve just had a very positive meeting in Paris with the IMS QTI Working Group. We discussed many things about the specification and documentation and a list of elements for the Main Profile was suggested. Lively discussion revealed that there are many interpretations of the term “profile”, depending on who is talking! There are, in fact a number of profiles under construction, so the requirement for more than one profile to be described, implemented by more than one set of tools and demonstrated is sure to be fulfilled.

Support

QTI Roadshows

The QTI-IPS project is now moving into the support stage of its activity, beginning with yesterday’s Roadshow at Strathclyde University. We had a full day – a very busy day – in which we

• Looked at the reasons for using standards for educational resources, particularly questions and tests,


• Showed some sample questions and tests which take e-assessment a good bit further than the normal MCQ-MAQ-text input,


• Demoed lots of tools for rendering, editing and converting questions and tests, and had participants trying them out in a long and lively hands-on session,


• Discussed the issues around QTI and uptake

Altogether, it was a very useful meeting, certainly from our point of view, and the feedback from delegates was generally very positive, but coloured by the financial constraints within HE and FE.

The second of these Roadshows will be the pre-conference workshop at the CAA conference in Southampton on July 4th, see http://jisc.cetis.ac.uk/events/.

Website

The website is taking shape in the virtual machine, and is visible at http://cloud.niallbarr.me.uk/.

Currently it has

• Two of the renderers – MathAssessEngine and JAssess – available for trial runs of questions,


• A Moodle instance which will soon have a course with sample tests for different disciplines,


• A link to the Eqiat question editor, developed at the University of Southampton under the EASiHE project



Coming soon





• SToMP II, a QTI 2.1 implementation written in PHP,


• QTItalk interactive QTI 2.1 documentation,


• QTICat which can take a set of questions and report on which renderers are able to run them,


• Links to more editors, with a bit of info about what they do best,


• Links to conversion tools – these will convert batches of questions at the same time,


• A growing library of examples of items and tests, demonstrating the specification - we still have to annotate these...


• The FAQ collection – currently under construction,


• Repositories – Minibix





Mailing Lists and Forums




The best places to ask questions about QTI are the mailing list at ims-qti@lists.ucles.org.uk and on the IMS Forum. Members of the team are keeping an eye on these lists and taking an active part in responding to queries.

Initial Activities and Dates for your Diary

The QTI-IPS team had its first face-to-face meeting in Glasgow on 06/04/2011. We were able to spend some time on the Project Plan, particularly the workpackages, with the aim of ensuring that all the tasks we have said we will do are covered by the plan. We shall have plenty to do, but we can draw on resources already in existence from previous projects and our ongoing activities. In particular we have:

• 3 different renderers for QTI: MathAssessEngine, JAssess and SToMPII


• The Spectatus test authoring application, which will be extended for packaging items and tests


• QTICat for profile building


• QTITalk for documentation


• A growing collection of items and tests, demonstrating the specification


• Contact with and support from colleagues in the wider QTI community and beyond.

We’ll be using a Virtual Machine (VM) to bundle all the tools, examples and other facilities, prior to deployment on the Amazon Cloud. The renderers are currently being brushed up ready for use in the VM and the other tools are under construction.
Meanwhile, work on the profiling aspect of the project is under way, with a list of elements and attributes for the profile in preparation, along with some examples to demonstrate the profile.

We have already had some support contacts through the public lists, and of course we expect that this aspect of the project will increase as time goes on. There will be two QTI Roadshow events in the early summer – one at Strathclyde on June 2nd and another at Kingston around the same time. These events will give users an overview of the capabilities of QTIv2.1 and the renderers and other tools and will enable participants to talk to the team about adopting QTI in their teaching.

QTI-IPS Project Starts

This is the Blog for the QTI-IPS project, funded by JISC, which began on 14th March and will continue until mid-September 2011.

To enable QTIv2.1 to become a final specification, at least two profiles must be specified, implemented and documented.

Interoperability with the first, simple CC QTIv2.1 profile has been publicly demonstrated with twelve different tools during the IMS quarterly held in Koblenz, Germany, August 2010, and it is intended for inclusion in a future version of Common Cartridge. Four of the QTIv2.1 renderers were initially JISC-sponsored projects.

The task of specifying, implementing and demonstrating the “Main Profile”, which covers the functionality that is used by as wide a set of stakeholders as possible, remains. This project will create and document a demonstration set of items and tests and existing rendering and authoring tools will be deployed to provide demonstration and code-testing facilities for the profile.

The project will also facilitate the involvement of other communities in QTI development, implementation and use, by ensuring that expert advice and practical help is at hand for those wishing to convert an existing question and test bank or author a new collection. In addition, it will ensure that a rich and representative set of example questions is available with the specification.

If you’re a QTI developer you are welcome to get in touch via :
http://www.imsglobal.org/community/forum/categories.cfm?catid=52 – register to post messages.
IMS-QTI@lists.ucles.org.uk (Sign up for this list at http://lists.ucles.org.uk/lists/listinfo/ims-qti )