Generation of E-Learning Exams in R for Moodle, OLAT, etc.

(Guest post by Achim Zeileis)
Development of the R package exams for automatic generation of (statistical) exams in R started in 2006 and version 1 was published in JSS by Grün and Zeileis (2009). It was based on standalone Sweave exercises, that can be combined into exams, and then rendered into different kinds of PDF output (exams, solutions, self-study materials, etc.). Now, a major revision of the package has been released that extends the capabilities and adds support for learning management systems. It is still based on the same type of
Sweave files for each exercise but can also render them into output formats like HTML (with various options for displaying mathematical content) and XML specifications for online exams in learning management systems such as Moodle or OLAT. Supplementary files such as graphics or data are
handled automatically. Here, I give a brief overview of the new capabilities. A detailed discussion is in the working paper by Zeileis, Umlauf, and Leisch (2012) that is also contained in the package as a vignette.

The basic idea…

…is to have standard standalone Sweave files for each exercise. These typically comprise (1) R code chunks (as usual within <<>>= and @) for random data generation. (2) Question and solution descriptions contained in LaTeX environments of corresponding names, typically with Sexpr{} for including data
for the question. (3) Metainformation about the exercise type (numeric, multiple choice, …), its correct solution etc. Then many different replications of (collections of) such an exercise can be easily created by repeating the following four steps:
Weaving the exercise (i.e., executing the R code for random data generation and embedding that into the text), reading the resulting LaTeX into R, transforming the LaTeX to another format such as HTML (if necessary), writing the corresponding output files, e.g., in PDF, HTML, XML, …
The package provides convenience interfaces exams2pdf(), exams2html()exams2moodle(), and exams2qti12() (for OLAT) that directly carry out all of these steps but the package also provides the underlying building blocks such that new interfaces can be easily created.

A simple example

To get a quick impression of the look and feel of the package, consider the following simple example: computation of a t statistic from given mean, variance, and sample size. The corresponding Sweave template tstat.Rnw is provided within the package. After installing the package, you can produce random versions of this exercise in PDF and HTML by running the following code:


The former needs a working LaTeX installation and then pops up a PDF with the exercise.
The latter does not need LaTeX but just the R package tth (interfacing Ian Hutchinson’s wonderful TtH
TeX-to-HTML converter) and then pops up the HTML with the exercise in a browser.
MathML support is needed for this version of the HTML exercise, e.g., by using the Firefox browser.

How it works

The tstat.Rnw is a relatively simple Sweave file: It first draws some random data and then uses the textbook formula to compute the correct t statistic. Subsequently, there are two LaTeX environments with the {question} and {solution}, respectively. Finally, some simple metainformation is provided
in LaTeX style (but commented):

n n}}}
    = frac{Sexpr{Mean} - Sexpr{mu}}{sqrt{frac{Sexpr{Var}}{Sexpr{n}}}}
    = Sexpr{tstat}.
  The absolute value of the $t$~test statistic is thus equal to
  $Sexpr{format(abs(tstat), nsmall = 3)}$.

%% extype{num}
%% exsolution{Sexpr{format(abs(tstat), nsmall = 3)}}
%% exname{t statistic}
%% extol{0.01}

After running Sweave() on it, the resulting LaTeX code has the random numbers filled in and the data-generating code omitted:

  A machine fills milk into $500$ml packages. It is suspected that the
  machine is not working correctly and that the amount of milk filled differs
  from the setpoint $mu_0 = 500$. A sample of $226$ packages
  filled by the machine are collected. The sample mean $bar{y}$ is equal to
  $517.2$ and the sample variance $s^2_{n-1}$ is equal to $262.56$.

  Test the hypothesis that the amount filled corresponds on average to the
  setpoint. What is the absolute value of the $t$~test statistic?

  The $t$~test statistic is calculated by:
    t & = & frac{bar y - mu_0}{sqrt{frac{s^2_{n-1}}{n}}}
    = frac{517.2 - 500}{sqrt{frac{262.56}{226}}}
    = 15.958.
  The absolute value of the $t$~test statistic is thus equal to

%% extype{num}
%% exsolution{15.958}
%% exname{t statistic}
%% extol{0.01}

The LaTeX code is then read into R, can be transformed into HTML using tth(), embedded into a simple HTML file, and then shown in a browser:


Support for learning management systems

The same HTML code shown in the example above can also be embedded into learning management
systems. These typically have XML-based exchange formats, e.g., Moodle uses its own Moodle XML
while OLAT and several other systems use the international IMS QTI 1.2 standard. The package provides functions exams2moodle() and exams2qti12() which provide output files that can be easily imported into Moodle and OLAT, respectively.
As a result, the tstat.Rnw looks in Moodle like this:


Note that this now has a field for entering the answer (and only after that the correct solution
can optionally be shown).

To generate a Moodle quiz with three random replications for five exercises drawn from collections
of different templates, consider the following example (based on templates provided within the

myexam <- list(
  c("confint", "ttest", "tstat"),
  c("anova", "regression"),
exams2moodle(myexam, n = 3)

This produces a file moodlequiz.xml that can be imported into the question bank in Moodle. (In this step, the HTML markup for formulas and graphics is not formatted by Moodle but don't worry: the final quiz will render correctly.) Subsequently, all replications of each exercise can be added as "random" questions into the quiz. Some more configuration of the quiz as a whole can then be done in Moodle but otherwise everything else is ready for the participants of the quiz.

If you want to use this for your own course...

  • You need an installation of a suitable learning management system but many universities have
    support for this anyway.
  • You need to build your collection of Sweave exercise templates. This is, of course,
    the most work. However, the package provides a suite of examples that can be helpful as
    starting points.
  • You need to export the exams/quizzes/assessments from R to the correct format for the
    learning management system. Moodle and OLAT are directly supported. But through QTI 1.2
    other systems could, in principle, be interfaced as well. We haven't tested this yet, though.

More details are provided in the working paper and we also provide a support forum on R-Forge if you experience any problems etc.

5 thoughts on “Generation of E-Learning Exams in R for Moodle, OLAT, etc.”

  1. Hello dear Achim,
    Thank you for your guest post, it is very interesting!

    A question: what are your thoughts on how packages like {knitr} and {Shiny} might be used to extend the work you have done?

  2. First, thanks for hosting this guest post, Tal!

    Second as for your questions concerning {knitr} and {Shiny}:

    The default function for weaving exercise files in {exams} is Sweave(). However, the design of the package is modular enough that other weaving functions could also be plugged in. So far Sweave() was enough for everything we wanted to do and we didn’t need the additional features of knitr. However, if someone else wants to play with
    this, the package’s building blocks can be easily cominbed to do so. See the paper/vignette for more details.

    As for Shiny: The way we conducted exams so far was either written exams
    (printed from PDF) or online exams conducted in an LMS (our university uses OLAT and Fritz’ university uses Moodle). And I think none of these could be easily combined with the Shiny features, so we didn’t think about this. If someone else sets up his/her own exam server than can easily leverage Shiny infrastructure, I guess that many building blocks in the “exams” package could be used. And, of course, if someone wants to pursue that and needs further information/recommendations about {exams}, just let us know.

  3. Currently I’m using probsoln LaTex package in combination with Sweave capabilities to produce random exams. It’s not difficult use them. probsoln gives excelent support for exams generation including types of organization of the answers (after exercise, at the end, etc), types of question (multiple choice, short and long answers, numeric, true or false). There is also types of including schemes like random, random on selected or selected exercices from a base file containing exercises. They are very useful to me. I think that merging the capabilities of probsoln with exams should be wonderful. Congratulations!

  4. I didn’t know the probsoln package before but just had a quick look. I think it shouldn’t be too hard to set up an exams2probsoln() function. The idea would be to call xexams(file, n) to produce a list of n random versions of an Sweave file and read it into R. Then, one could write this into probsoln databases and finally produce PDFs from them. Look at the output of xexams(“tstat”, n = 2) or xexams(“ttest”, n = 2) for examples of a numeric and a multiple-choice exercise, respectively. Writing these into probsoln files shouldn’t be too hard but, of course, the devil is probably in the detail (as always).

    However, if you want to create PDF files anyway, then it may be simpler to not have to go through a finite database but simply create a new exercise on the fly every time you need one. See the examples in the old Grün & Zeileis (2009) paper on how to create both exams and corresponding solutions from the same exercise templates.

  5. Moodle can easily be installed on any computer that runs on PHP. The system may be also extended with modules for quizzes, social learning, assignments and grading. It is also great for certification in an engaging way.

Leave a Reply to Walmes Zeviani Cancel reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.