(Guest post by Achim Zeileis)

Development of the R package exams for automatic generation of (statistical) exams in R started in 2006 and version 1 was published in JSS by Grün and Zeileis (2009). It was based on standalone `Sweave `exercises, that can be combined into exams, and then rendered into different kinds of PDF output (exams, solutions, self-study materials, etc.). Now, a major revision of the package has been released that extends the capabilities and adds support for learning management systems. It is still based on the *same* type of

`Sweave` files for each exercise but can also render them into output formats like HTML (with various options for displaying mathematical content) and XML specifications for online exams in learning management systems such as Moodle or OLAT. Supplementary files such as graphics or data are

handled automatically. Here, I give a brief overview of the new capabilities. A detailed discussion is in the working paper by Zeileis, Umlauf, and Leisch (2012) that is also contained in the package as a vignette.

### The basic idea…

…is to have standard standalone `Sweave` files for each exercise. These typically comprise (1) R code chunks (as usual within `<<>>=` and `@`) for random data generation. (2) Question and solution descriptions contained in LaTeX environments of corresponding names, typically with `\Sexpr{}` for including data

for the question. (3) Metainformation about the exercise type (numeric, multiple choice, …), its correct solution etc. Then many different replications of (collections of) such an exercise can be easily created by repeating the following four steps:

*Weaving* the exercise (i.e., executing the R code for random data generation and embedding that into the text), *reading* the resulting LaTeX into R, *transforming* the LaTeX to another format such as HTML (if necessary), *writing* the corresponding output files, e.g., in PDF, HTML, XML, …

The package provides convenience interfaces `exams2pdf()`, `exams2html()`, `exams2moodle()`, and `exams2qti12()` (for OLAT) that directly carry out all of these steps but the package also provides the underlying building blocks such that new interfaces can be easily created.

### A simple example

To get a quick impression of the look and feel of the package, consider the following simple example: computation of a *t* statistic from given mean, variance, and sample size. The corresponding `Sweave` template `tstat.Rnw` is provided within the package. After installing the package, you can produce random versions of this exercise in PDF and HTML by running the following code:

library("exams") exams2pdf("tstat.Rnw") exams2html("tstat.Rnw") |

The former needs a working LaTeX installation and then pops up a PDF with the exercise.

The latter does not need LaTeX but just the R package tth (interfacing Ian Hutchinson’s wonderful TtH

TeX-to-HTML converter) and then pops up the HTML with the exercise in a browser.

MathML support is needed for this version of the HTML exercise, e.g., by using the Firefox browser.

### How it works

The `tstat.Rnw` is a relatively simple `Sweave` file: It first draws some random data and then uses the textbook formula to compute the correct *t* statistic. Subsequently, there are two LaTeX environments with the `{question}` and `{solution}`, respectively. Finally, some simple metainformation is provided

in LaTeX style (but commented):

<>= ## DATA GENERATION n n}}} = \frac{\Sexpr{Mean} - \Sexpr{mu}}{\sqrt{\frac{\Sexpr{Var}}{\Sexpr{n}}}} = \Sexpr{tstat}. \end{eqnarray*} The absolute value of the $t$~test statistic is thus equal to $\Sexpr{format(abs(tstat), nsmall = 3)}$. \end{solution} %% META-INFORMATION %% \extype{num} %% \exsolution{\Sexpr{format(abs( tstat), nsmall = 3)}} %% \exname{t statistic} %% \extol{0.01}

After running `Sweave()` on it, the resulting LaTeX code has the random numbers filled in and the data-generating code omitted:

\begin{question} A machine fills milk into $500$ml packages. It is suspected that the machine is not working correctly and that the amount of milk filled differs from the setpoint $\mu_0 = 500$. A sample of $226$ packages filled by the machine are collected. The sample mean $\bar{y}$ is equal to $517.2$ and the sample variance $s^2_{n-1}$ is equal to $262.56$. Test the hypothesis that the amount filled corresponds on average to the setpoint. What is the absolute value of the $t$~test statistic? \end{question} \begin{solution} The $t$~test statistic is calculated by: \begin{eqnarray*} t & = & \frac{\bar y - \mu_0}{\sqrt{\frac{s^2_{n-1}}{n}}} = \frac{517.2 - 500}{\sqrt{\frac{262.56}{226}} } = 15.958. \end{eqnarray*} The absolute value of the $t$~test statistic is thus equal to $15.958$. \end{solution} %% META-INFORMATION %% \extype{num} %% \exsolution{15.958} %% \exname{t statistic} %% \extol{0.01}

The LaTeX code is then read into R, can be transformed into HTML using `tth()`, embedded into a simple HTML file, and then shown in a browser:

### Support for learning management systems

The same HTML code shown in the example above can also be embedded into learning management

systems. These typically have XML-based exchange formats, e.g., Moodle uses its own Moodle XML

while OLAT and several other systems use the international IMS QTI 1.2 standard. The package provides functions `exams2moodle()` and `exams2qti12() `which provide output files that can be easily imported into Moodle and OLAT, respectively.

As a result, the `tstat.Rnw` looks in Moodle like this:

Note that this now has a field for entering the answer (and only after that the correct solution

can optionally be shown).

To generate a Moodle quiz with three random replications for five exercises drawn from collections

of different templates, consider the following example (based on templates provided within the

package):

myexam <- list( "boxplots", c("confint", "ttest", "tstat"), c("anova", "regression"), "scatterplot", "relfreq") exams2moodle(myexam, n = 3) |

This produces a file `moodlequiz.xml` that can be imported into the question bank in Moodle. (In this step, the HTML markup for formulas and graphics is not formatted by Moodle but don’t worry: the final quiz will render correctly.) Subsequently, all replications of each exercise can be added as “random” questions into the quiz. Some more configuration of the quiz as a whole can then be done in Moodle but otherwise everything else is ready for the participants of the quiz.

### If you want to use this for your own course…

- You need an installation of a suitable learning management system but many universities have

support for this anyway. - You need to build your collection of
`Sweave`exercise templates. This is, of course,

the most work. However, the package provides a suite of examples that can be helpful as

starting points. - You need to export the exams/quizzes/assessments from R to the correct format for the

learning management system. Moodle and OLAT are directly supported. But through QTI 1.2

other systems could, in principle, be interfaced as well. We haven’t tested this yet, though.

More details are provided in the working paper and we also provide a support forum on R-Forge if you experience any problems etc.

Hello dear Achim,

Thank you for your guest post, it is very interesting!

A question: what are your thoughts on how packages like {knitr} and {Shiny} might be used to extend the work you have done?

First, thanks for hosting this guest post, Tal!

Second as for your questions concerning {knitr} and {Shiny}:

The default function for weaving exercise files in {exams} is Sweave(). However, the design of the package is modular enough that other weaving functions could also be plugged in. So far Sweave() was enough for everything we wanted to do and we didn’t need the additional features of knitr. However, if someone else wants to play with

this, the package’s building blocks can be easily cominbed to do so. See the paper/vignette for more details.

As for Shiny: The way we conducted exams so far was either written exams

(printed from PDF) or online exams conducted in an LMS (our university uses OLAT and Fritz’ university uses Moodle). And I think none of these could be easily combined with the Shiny features, so we didn’t think about this. If someone else sets up his/her own exam server than can easily leverage Shiny infrastructure, I guess that many building blocks in the “exams” package could be used. And, of course, if someone wants to pursue that and needs further information/recommendations about {exams}, just let us know.

Currently I’m using probsoln LaTex package in combination with Sweave capabilities to produce random exams. It’s not difficult use them. probsoln gives excelent support for exams generation including types of organization of the answers (after exercise, at the end, etc), types of question (multiple choice, short and long answers, numeric, true or false). There is also types of including schemes like random, random on selected or selected exercices from a base file containing exercises. They are very useful to me. I think that merging the capabilities of probsoln with exams should be wonderful. Congratulations!

I didn’t know the probsoln package before but just had a quick look. I think it shouldn’t be too hard to set up an exams2probsoln() function. The idea would be to call xexams(file, n) to produce a list of n random versions of an Sweave file and read it into R. Then, one could write this into probsoln databases and finally produce PDFs from them. Look at the output of xexams(“tstat”, n = 2) or xexams(“ttest”, n = 2) for examples of a numeric and a multiple-choice exercise, respectively. Writing these into probsoln files shouldn’t be too hard but, of course, the devil is probably in the detail (as always).

However, if you want to create PDF files anyway, then it may be simpler to not have to go through a finite database but simply create a new exercise on the fly every time you need one. See the examples in the old Grün & Zeileis (2009) paper on how to create both exams and corresponding solutions from the same exercise templates.

Moodle can easily be installed on any computer that runs on PHP. The system may be also extended with modules for quizzes, social learning, assignments and grading. It is also great for certification in an engaging way.