Repeated measures ANOVA with R (functions and tutorials)

Repeated measures ANOVA is a common task for the data analyst.

There are (at least) two ways of performing “repeated measures ANOVA” using R but none is really trivial, and each way has it’s own complication/pitfalls (explanation/solution to which I was usually able to find through searching in the R-help mailing list).

So for future reference, I am starting this page to document links I find to tutorials, explanations (and troubleshooting) of “repeated measure ANOVA” done with R

Functions and packages

(I suggest using the tutorials supplied bellow for how to use these functions)

  • aov {stats} – offers SS type I repeated measures anova, by a call to lm for each stratum. A short example is given in the ?aov help file
  • Anova {car} – Calculates type-II or type-III analysis-of-variance tables for model objects produced by lm, and for various other object. The ?Anova help file offers an example for how to use this for repeated measures
  • ezANOVA {ez} – This function provides easy analysis of data from factorial experiments, including purely within-Ss designs (a.k.a. “repeated measures”), purely between-Ss designs, and mixed within-and-between-Ss designs, yielding ANOVA results and assumption checks. It is a wrapper of the Anova {car} function, and is easier to use. The ez package also offers the functions ezPlot and ezStats to give plot and statistics of the ANOVA analysis. The ?ezANOVA help file gives a good demonstration for the functions use (My thanks goes to Matthew Finkbe for letting me know about this cool package)
  • friedman.test {stats} – Performs a Friedman rank sum test with unreplicated blocked data. That is, a non-parametric one-way repeated measures anova. I also wrote a wrapper function to perform and plot a post-hoc analysis on the friedman test results
  • Non parametric multi way repeated measures anova – I believe such a function could be developed based on the Proportional Odds Model, maybe using the {repolr} or the {ordinal} packages. But I still didn’t come across any function that implements these models (if you do – please let me know in the comments).
  • Repeated measures, non-parametric, multivariate analysis of variance – as far as I know, such a method is not currently available in R.  There is, however, the Analysis of similarities (ANOSIM) analysis which provides a way to test statistically whether there is a significantdifference between two or more groups of sampling units.  Is is available in the {vegan} package through the “anosim” function.  There is also a tutorial and a relevant published paper.

Good Tutorials

Troubelshooting

Unbalanced design
Unbalanced design doesn’t work when doing repeated measures ANOVA with aov, it just doesn’t. This situation occurs if there are missing values in the data or that the data is not from a fully balanced design. The way this will show up in your output is that you will see the between subject section showing withing subject variables.

A solution for this might be to use the Anova function from library car with parameter type=”III”. But before doing that, first make sure you understand the difference between SS type I, II and III. Here is a good tutorial for helping you out with that.
By the way, these links are also useful in case you want to do a simple two way ANOVA for unbalanced design

I will “later” add R-help mailing list discussions that I found helpful on the subject.

If you come across good resources, please let me know about them in the comments.

  • Nicholas

    Howdy,
    I am going to get a reputation as the disgruntled nay sayer. Two comments:
    1) Type III sums of squares should be avoided for the very compelling reasons stated in the following document, which should be required reading for all Splus/R users
    http://www.stats.ox.ac.uk/pub/MASS3/Exegeses.pdf

    2) Repeated measures are handled much better within the linear mixed model framework. Unbalanced designs are handled better, with the note that for small samples the variances may not be estimated with any precision. But this would bias classical ANOVA in the unbalanced case as well.

    –The grumpy old man

    • http://www.talgalili.com Tal Galili

      Hello Nicholas,
      As always – thank you for a very informative reply.

      I am joining Patrick with asking for any suggestions for good resources for performing repeated measures analysis with linear mixed models.

      Best,
      Tal

  • Patrick

    Thanks for posting this Tal, I’ve tried to poke around several times to find resources on repeated measure analysis with R and haven’t had much luck.

    Nicholas, would you have any links to a good resource/tutorial on performing repeated measure analysis with the linear mixed model framework?

    Cheers,

    Pat

  • Dror

    Thanks Tal for addressing this important (to me, at least :-) and yet still somewhat tricky (at least to the less savvy R user).

    I’ve found the r-tutorial of Jonathan Baron & Yuelin Li helpful: http://www.psych.upenn.edu/~baron/rpsych/rpsych.html
    Section 6.9 addresses repeated measure anova.
    And there is the (unresolved?) issue of simple-main-effects. for example: http://tinyurl.com/ygkfrty

    All the best,
    dror

    • http://www.talgalili.com Tal Galili

      Thanks Dror,
      I am glad this post was of help to you :)
      I didn’t know about that resource you mentioned – I added it to the list of links – thanks.

      Regarding Or’s question and the issue of “simple-main-effects”, I remember meeting him in the Israeli R tutorial session about a month ago. He asked me about it, I remember understanding better what he wanted, and invited him to e-mail me with a description of it so I might help him. He didn’t do so, so I dropped the subject.

      Cheers,
      Tal

  • matthew

    These links are great! But have a look at the ez package; it’s all I use for repeated measures anymore.

    http://cran.r-project.org/web/packages/ez/index.html

    • http://www.talgalili.com Tal Galili

      Hi Matthew,
      Thank you so much for the heads up on this cool package.
      Your comment inspired me to extend my post to include reference to it (while also giving you due credit).

      Best,
      Tal

  • Alisa Coffin

    Thanks for posting these resources. I’ve been reading the section in Agresti & Finlay’s statistics textbook on using repeated measures ANOVA (they use SAS and SPSS). But, I only have access to R and need some guidance. This is very helpful, including all the comments.

  • Pingback: New edition of R Companion to Applied Regression (by John Fox and Sandy Weisberg) | R-statistics blog()

  • Pingback: Free statistics e-books for download | R-statistics blog()

  • Jubei

    Thank you!

  • İrfan Emrah Kanat

    Very helpful thank you

  • michael

    great man!!!! I just added this notes to my archive

  • melanie

    Thank you for the links, here is another simpler piece which made me go “aaah”.http://www.stat.umn.edu/geyer/5102/examp/dummy.html

  • junzy

    I have tested differences between treatments using ANOSIM in R. But, how can I get the results of multiply comparison? Thanks a lot.

  • Probiers Malaus

    Tal, I’ve followed the link form StackExchange’s CrossValidated to here – don’t have a working solution to offer, so I’m not going to comment there.

    However: this might be interesting.
    Oliver-Rodríguez, J. C. and Wang, X. T. (2013), Non-parametric three-way
    mixed ANOVA with aligned rank tests. British Journal of Mathematical
    and Statistical Psychology. doi:10.1111/bmsp.12031
    (Not implemented in R, currently, I guess. Also seen no discussions about it.)

    I also tried out this, which comes with handy R code:
    Herberich E., Sikorski J., Hothorn T. (2010), A Robust Procedure for
    Comparing Multiple Means under Heteroscedasticity in Unbalanced Designs.
    PLoS ONE 5(3): e9788. doi:10.1371/journal.pone.0009788

    Still, I am not really convinced. Seems like there’s quite some demand for
    something like this, but it’s mathematically quite challenging. Out of
    my league, sadly.

    FTR, as I often see people recommend adonis{vegan} for this type of analysis: these are not the droids we are looking for. Check out Kay Cichini’s blog for repeated measures and permutation trouble:

    http://thebiobucket.blogspot.de/2011/04/repeat-measure-adonis-lately-i-had-to.html
    – again, for complicated designs, this is a pain and really challenging.

    • http://www.r-statistics.com/ Tal Galili

      Thank you for the detailed reply Probiers.