• Gelman, A., & Hill, J. (2006). Data analysis using regression and multilevel/hierarchical models. Cambridge University Press.

• Hastie, T., Tibshirani, R., Friedman, J., Hastie, T., Friedman, J., & Tibshirani, R. (2009). The elements of statistical learning (Vol. 2, No. 1). New York: Springer.

• Russell, M. A. (2013). Mining the Social Web: Data Mining Facebook, Twitter, LinkedIn, Google+, GitHub, and More. O’Reilly Media, Inc.

• McDowell, G. L. (2013). Cracking the Coding Interview: 150 Programming Questions and Solutions. CareerCup.

• Chang, W. (2012). R graphics cookbook. O’Reilly Media, Inc.

I actually read through Winston’s cookbook before Insight, but it’s been an invaluable resource. Why write 20 lines of matplotlib or R base graphics when you can accomplish a better graph using 5 lines of ggplot2?

What is recursion?

Recursive functions call themselves. That is, they break down the problem into the smallest possible components and the function() calls itself within the original function() on each of the smaller components. Afterward, the results are put together to solve the original problem. Let’s take a look at more concrete examples.

Quicksort

A technical interview will usually ask you to implement some type of sorting function that can be solved using a recursive algorithm. Let’s try implementing quicksort in R:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

#!/usr/bin/env Rscript
# Author:  Jason A. French

quickSort <- function(vect) {
  # Args:
  #  vect: Numeric Vector
  
  # Stop if vector has length of 1
  if (length(vect) <= 1) {
      return(vect)
  }
  # Pick an element from the vector
  element <- vect[1]
  partition <- vect[-1]
  # Reorder vector so that integers less than element
  # come before, and all integers greater come after.
  v1 <- partition[partition < element]
  v2 <- partition[partition >= element]
  # Recursively apply steps to smaller vectors.
  v1 <- quickSort(v1)
  v2 <- quickSort(v2)
  return(c(v1, element, v2))
}

1
2

quickSort(c(4, 65, 2, -31, 0, 99, 83, 782, 1))
[1] -31   0   1   2   4  65  83  99 782

Merge Sort

A second sorting algorithm that we can implement using recursion is the Merge Sort. Sorting algorithms are important because they differ in their speed, depending on the nature of the input data. In the case of merge sort, the worst possible outcome for the time it takes to sort the data is n * log(n), where n is the length of your list of numbers.

Like Quicksort, we’re splitting the vector into smaller sub-vectors by halving it until each vector has just one integer. We then merge the vectors back together, this time in order. Let’s tackle this algorithm in R as well.

I found that Rosetta Code already provides a good example, so below is the their public domain code with my comments added to the code for explanation.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61

#!/usr/bin/env Rscript
# http://rosettacode.org/wiki/Sorting_algorithms/Merge_sort#R

mergesort <- function(m)

{
   merge_ <- function(left, right)
   # Recursive function to compare and append values in order
   {
      # Create a list to hold the results
      result <- c()
      # This is our stop condition. While left and right contain 
      # a value, compare them
      while(length(left) > 0 && length(right) > 0)
      {
           # If left is less than or equal to right,
           # add it to the result list
         if(left[1] <= right[1])
         {
            result <- c(result, left[1])
            # Remove the value from the list
            left <- left[-1]
         } else
         {
            # When right is less than or equal to left,
            # add it to the result.
            result <- c(result, right[1])
            # Remove the appended integer from the list.
            right <- right[-1]
         }
      }
      # Keep appending the values to the result while left and right
      # exist.
      if(length(left) > 0) result <- c(result, left)
      if(length(right) > 0) result <- c(result, right)
      result
   }

   # Below is our stop condition for the mergesort function.
   # When the length of the vector is 1, just return the integer. 
   len <- length(m)
   if(len <= 1) m else
   {
      # Otherwise keep dividing the vector into two halves.
      middle <- length(m) / 2
      # Add every integer from 1 to the middle to the left
      left <- m[1:floor(middle)]
      right <- m[floor(middle+1):len]
      # Recursively call mergesort() on the left and right halves.
      left <- mergesort(left)
      right <- mergesort(right)
      # Order and combine the results.
      if(left[length(left)] <= right[1])
      {
         c(left, right)
      } else
      {
         merge_(left, right)
      }
   }
}

Memory Efficiency

Relational databases, such as MySQL, organize data into tables (like a spreadsheet) and can link values in tables to each other. Generally speaking, they are better at handling large datasets and are more efficient at storing and accessing data than CSVs due to compression and indexing. The data stay in the MySQL database until accessed via a query, which is different than how R approaches data.frames and CSVs. When accessing data stored in a data.frame or CSV file in R, the data must all fit in memory. However, this becomes a problem if when using a large dataset or if you’re cursed with an older computer with <= 4GB of RAM. In these cases, every time you load your dataset or do a memory-intensive operation (e.g., polychoric correlations) your computer will slow to a painful crawl as your hard drives grind while your operating system switches to using virtual memory or swap space, rather than RAM. You could get around this by using random sampling to create a smaller subset (or multiple subsets if you want to bootstrap), but you can also use this technique on a SQL database. The advantage is that SQL databases only load the data you’re working with into your local machine’s RAM when you SELECT the data you need - leaving plenty of memory for the actual analysis.

Safety and Security

Two additional reasons I prefer SQL databases are safety and security. Services such as Amazon’s RDS (i.e., Relational Database Service) offer frequent backups and easy replication across instances. If the local machine dies, the data are safe and uncorrupted. If I were storing data in an .Rdata file, I’d have to rely on my OS (e.g., Time Machine) or a cloud-based storage solution (e.g., Dropbox or CrashPlan) to backup my data. Worse, if I were storing my data in an Excel database and Excel crashed, I may risk losing years of progress.

Using MySQL, I can give a collaborator access to only needed portions of the data by locking down their SQL user to certain tables and operations. By limiting access, we limit risk of corruption and overwriting years of progress.

Scalable as the need grows

Last, I find databases such as MySQL to be more scalable due to cloud computing. In the case of using Amazon’s RDS, you can buy as much storage or bandwidth as you need to complete your analysis, dissertation, or build your web app. If I need multiple nodes or machines to analyze the data, I don’t need to keep a copy of the data on each machine. I simply pass the SQL authentication parameters to each machine and the data is accessed from a central location.

So let’s begin by setting up a Free-Tier RDS service on Amazon and then move some data from R to the RDS database.

Setting up an RDS Instance

Amazon currently offers a Free Tier for their RDS storage. It offers:

• Enough hours to run a DB Instance continuously each month,
• 20 GB of database storage,
• 10 million I/O operations, and
• automated database backups.

To setup an RDS instance:

1. Sign into the AWS Management Console.
2. Select ‘RDS’ under the ‘Database’ group.
3. Click ‘Launch DB Instance’.
4. Select ‘MySQL’ for the database engine.
5. Select ‘No’ when asked if you intend to use the database for production purposes. Accidentally selecting ‘Yes’ could result in hefty fees.
6. For ‘Instance Class’, select ‘db.t1.micro’, ‘No’ for ‘Multi-AZ Deployment’, and 20 GB for ‘Allocated Storage’.
7. Click ‘Next’ and launch your RDS instance.

At this point, you should have a blank RDS database with a master username and password combination. Be sure to test that you can connect - I suggest using Sequel Pro. If you can’t connect, you may need to open up port 3306 in your EC2 Security Group.

Click on your new RDS Instance to get the hostname to connect to and note the database name:

Fire up Sequel Pro and test out your new database. If you want to connect using SSL, you’ll need the SSH key you generated when you first created your EC2 instance (not covered here).

Installing the RMySQL package

Previously, I found installing RMySQL to be very difficult when I was using MacPorts. However, having switched to Homebrew on my new machine, RMySQL found all the MySQL headers and libraries as Homebrew uses the /usr/local directory to store your software, which is already in your shell $PATH.

Before compiling RMySQL, first compile MySQL using Homebrew:

1
2
3
4
5

# If you haven't already, install Homebrew:
# ruby -e "$(curl -fsSL https://raw.github.com/Homebrew/homebrew/go/install)"

# Install MySQL
brew install mysql

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

==> Downloading https://downloads.sf.net/project/machomebrew/Bottles/mysql-5.6.1
Already downloaded: /Library/Caches/Homebrew/mysql-5.6.19.mavericks.bottle.tar.gz
==> Pouring mysql-5.6.19.mavericks.bottle.tar.gz
==> Caveats
A "/etc/my.cnf" from another install may interfere with a Homebrew-built
server starting up correctly.

To connect:
    mysql -uroot

To have launchd start mysql at login:
    ln -sfv /usr/local/opt/mysql/*.plist ~/Library/LaunchAgents
Then to load mysql now:
    launchctl load ~/Library/LaunchAgents/homebrew.mxcl.mysql.plist
Or, if you don't want/need launchctl, you can just run:
    mysql.server start
==> Summary
🍺  /usr/local/Cellar/mysql/5.6.19: 9536 files, 338M

Next, compile RMySQL from source:

1

install.packages('RMySQL', type = 'source')

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58

trying URL 'http://cran.rstudio.com/src/contrib/RMySQL_0.9-3.tar.gz'
Content type 'application/x-gzip' length 165363 bytes (161 Kb)
opened URL
==================================================
downloaded 161 Kb

* installing *source* package ‘RMySQL’ ...
** package ‘RMySQL’ successfully unpacked and MD5 sums checked
checking for gcc... gcc
checking for C compiler default output file name... a.out
checking whether the C compiler works... yes
checking whether we are cross compiling... no
checking for suffix of executables...
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ANSI C... none needed
checking how to run the C preprocessor... gcc -E
checking for compress in -lz... yes
checking for getopt_long in -lc... yes
checking for mysql_init in -lmysqlclient... yes
checking for egrep... grep -E
checking for ANSI C header files...
rm: conftest.dSYM: is a directory
rm: conftest.dSYM: is a directory
yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking mysql.h usability... no
checking mysql.h presence... no
checking for mysql.h... no
checking /usr/local/include/mysql/mysql.h usability... yes
checking /usr/local/include/mysql/mysql.h presence... yes
checking for /usr/local/include/mysql/mysql.h... yes
configure: creating ./config.status
config.status: creating src/Makevars
** libs
clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG -I/usr/local/include/mysql -I/usr/local/include -I/usr/local/include/freetype2 -I/opt/X11/include    -fPIC  -Wall -mtune=core2 -g -O2  -c RS-DBI.c -o RS-DBI.o
clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG -I/usr/local/include/mysql -I/usr/local/include -I/usr/local/include/freetype2 -I/opt/X11/include    -fPIC  -Wall -mtune=core2 -g -O2  -c RS-MySQL.c -o RS-MySQL.o
clang -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/usr/local/lib -o RMySQL.so RS-DBI.o RS-MySQL.o -lmysqlclient -lz -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
installing to /Library/Frameworks/R.framework/Versions/3.1/Resources/library/RMySQL/libs
** R
** inst
** preparing package for lazy loading
Creating a generic function for ‘format’ from package ‘base’ in package ‘RMySQL’
Creating a generic function for ‘print’ from package ‘base’ in package ‘RMySQL’
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded
* DONE (RMySQL)

Transferring Existing Data to/from MySQL

Having a database is useless unless we can easily convert our existing data.frames into MySQL tables. Let’s try this using the Thurstone dataset from the Revelle’s psych package:

1
2
3
4
5
6
7
8

library(RMySQL)
library(psych)
con <- dbConnect(MySQL(),
    user = 'RDSUser',
    password = 'YourPass',
    host = 'RDS Host',
    dbname='YourDB')
dbWriteTable(conn = con, name = 'Test', value = as.data.frame(Thurstone))

There! Now we have transferred our data.frame to our SQL database.

Similarly, if you want to read data from MySQL to R, we can use the dbReadTable() function, which returns a data.frame object. You can keep your data in MySQL and stream portions into R for specific analyses. Since dbReadTable outputs a data.frame, we can use this command in place of a data.frame.

Let’s run an omega() factor analysis on the Thurstone table:

1

omega(dbReadTable(conn = con,name = 'Test'), title = "9 variables from Thurstone")

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53

Loading required package: MASS
Loading required package: GPArotation
Loading required package: parallel
9 variables from Thurstone
Call: omega(m = dbReadTable(conn = con, name = "Test"), title = "9 variables from Thurstone")
Alpha:                 0.89
G.6:                   0.91
Omega Hierarchical:    0.74
Omega H asymptotic:    0.79
Omega Total            0.93

Schmid Leiman Factor loadings greater than  0.2
                   g   F1*   F2*   F3*   h2   u2   p2
Sentences       0.71  0.57             0.82 0.18 0.61
Vocabulary      0.73  0.55             0.84 0.16 0.63
Sent_Completion 0.68  0.52             0.73 0.27 0.63
First_Letters   0.65        0.56       0.73 0.27 0.57
X4_Letter_Words 0.62        0.49       0.63 0.37 0.61
Suffixes        0.56        0.41       0.50 0.50 0.63
Letter_Series   0.59              0.61 0.72 0.28 0.48
Pedigrees       0.58  0.23        0.34 0.50 0.50 0.66
Letter_Group    0.54              0.46 0.53 0.47 0.56

With eigenvalues of:
   g  F1*  F2*  F3*
3.58 0.96 0.74 0.71

general/max  3.71   max/min =   1.35
mean percent general =  0.6    with sd =  0.05 and cv of  0.09
Explained Common Variance of the general factor =  0.6

The degrees of freedom are 12  and the fit is  0.01

The root mean square of the residuals is  0.01
The df corrected root mean square of the residuals is  0.01

Compare this with the adequacy of just a general factor and no group factors
The degrees of freedom for just the general factor are 27  and the fit is  1.48

The root mean square of the residuals is  0.14
The df corrected root mean square of the residuals is  0.16

Measures of factor score adequacy
                                                 g  F1*  F2*  F3*
Correlation of scores with factors            0.86 0.73 0.72 0.75
Multiple R square of scores with factors      0.74 0.54 0.52 0.56
Minimum correlation of factor score estimates 0.49 0.08 0.03 0.11

 Total, General and Subset omega for each subset
                                                 g  F1*  F2*  F3*
Omega total for total scores and subscales    0.93 0.92 0.83 0.79
Omega general for total scores and subscales  0.74 0.58 0.50 0.47
Omega group for total scores and subscales    0.16 0.35 0.32 0.32

In other analyses, you may only need a fraction of the MySQL table. In these cases, we’d want to use the dbGetQuery() command and issue a MySQL statement to SELECT and aggregate the information. Let’s get a vector of Sentences correlations that are greater than 0.5:

1

dbGetQuery(conn = con, statement = "SELECT Sentences FROM Test WHERE Sentences > 0.5;")

1
2
3
4
5

  Sentences
1     1.000
2     0.828
3     0.776
4     0.541

Safely storing credentials

It’s important to note that in the above example, although I passed the user and password parameters within my dbConnect() function, you would never want to do this in production code that you were sharing. It’s far better to store the credentials a text file where dbConnect(MySQL()) will look when you don’t pass any credentials:

1
2
3
4

user = youruser
password = yourpassword
host = yourhouse
dbname = yourdb

Useful Packages for Fast DB Operations

Ideally, we want to have one set of analysis code that is agnostic to how the data are stored. I suggest looking into Hadley Wickham’s new dplyr package (Github). dplyr is able to filter, sort, group, and summarize data quickly whether it is stored in a data.frame, data.table, or database. Database operations may not always be as fast as a data.table operation, but again, the advantage is that you don’t need to feed all of your data into memory.

Hadley provides a few vignettes for getting getting started with dplyr:

1
2

vignette("introduction", package = "dplyr")
vignette("databases", package = "dplyr")

Reproducible research is a phrase that describes an academic paper or manuscript that contains the code and data in addition to what is usually published - the researcher’s interpretation. In doing so, the experimental design and method of analysis is easily replicated by unaffiliated labs and critiqued by reviewers as the full analysis used to produce the results is submitted along with the final paper. One way of producing reproducible research is to use R code directly inside your LaTeX document. In order to faciliate the combination of statistical code and manuscript writing, two R packages in particular have arisen: Sweave and knitr. knitr is an R package designed as a replacement for Sweave, but both packages combine your R analysis with your LaTeX manuscript (i.e., knitr = R + LaTeX).

One advantage of knitr is that the researcher can easily create ANOVA and demographic tables directly from the data without messing around in Excel. However, as we’ll see, both knitr and Sweave can run into problems when formatting your table values to 2 decimal points. In this post, I’ll detail my proposed method of fixing that which can be applied to your entire mansucript by editing the beginning of your knitr preamble.

The basic example below contains the beginning of a hypothetical Methods section of a manuscript. We want to take the values from an R table, which has the breakdown of participants by gender and ethnicity, and display them as numbers in our manuscript.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48

\documentclass[12pt]{article}
\usepackage[sc]{mathpazo}
\usepackage[T1]{fontenc}
\usepackage{geometry}
\geometry{verbose,tmargin=2.5cm,bmargin=2.5cm,lmargin=2.5cm,rmargin=2.5cm}
\setcounter{secnumdepth}{2}
\setcounter{tocdepth}{2}
\usepackage{url}
\usepackage[unicode=true,pdfusetitle,
 bookmarks=true,bookmarksnumbered=true,bookmarksopen=true,bookmarksopenlevel=2,
 breaklinks=false,pdfborder={0 0 1},backref=false,colorlinks=false]
 {hyperref}
\hypersetup{
 pdfstartview={XYZ null null 1}}
\usepackage{breakurl}
\begin{document}

<<setup, include=FALSE, cache=FALSE>>=
library(knitr)
# set global chunk options
opts_chunk$set(fig.path='figure/minimal-', fig.align='center', fig.show='hold')
options(replace.assign=TRUE,width=90)
@

\title{A Minimal Demo of knitr}
\author{Jason A. French}
\maketitle


<<random-ethnicity, include=FALSE>>=
# Create data.frame of random ethnicities
x <- data.frame(Ethnicity = sample(as.factor(
  rep(x = c('White','African American','Asian American', 'Latino', 'Pacific Islander')
    )
  ),size = 100, replace = TRUE),
  Gender=rep(c('Male', 'Female'),100))
x.table <- table(x)
@

\section{Methods}
We recruited \Sexpr{sum(x.table)} university undergraduates from an introductory psychology class. Participants were drawn from various genders and ethnic groups across the Chicago area (see Table~\ref{tab:ethnicity})...

<<ethnicity-table, echo = FALSE, results = 'asis'>>=
library(xtable)
xtable(x.table, caption = 'Participant Ethnicities', label='tab:ethnicity')
@

\end{document}

As we see below, running the knit() command on our knitr manuscript inside R produces a regular LaTeX file that can be compiled with to a PDF using pdflatex or TeX Shop. Notice that the R table objects have been replaced with LaTeX tables.

1
2

library(knitr)
knit(input = 'knitr.Rnw', output = 'knitr.tex')

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63

\documentclass[12pt]{article}\usepackage[]{graphicx}\usepackage[]{color}
\makeatletter
\definecolor{fgcolor}{rgb}{0.345, 0.345, 0.345}
\newcommand{\hlnum}[1]{\textcolor[rgb]{0.686,0.059,0.569}{#1}}
\newcommand{\hlstr}[1]{\textcolor[rgb]{0.192,0.494,0.8}{#1}}
\newcommand{\hlcom}[1]{\textcolor[rgb]{0.678,0.584,0.686}{\textit{#1}}}
\newcommand{\hlopt}[1]{\textcolor[rgb]{0,0,0}{#1}}
\newcommand{\hlstd}[1]{\textcolor[rgb]{0.345,0.345,0.345}{#1}}
\newcommand{\hlkwa}[1]{\textcolor[rgb]{0.161,0.373,0.58}{\textbf{#1}}}
\newcommand{\hlkwb}[1]{\textcolor[rgb]{0.69,0.353,0.396}{#1}}
\newcommand{\hlkwc}[1]{\textcolor[rgb]{0.333,0.667,0.333}{#1}}
\newcommand{\hlkwd}[1]{\textcolor[rgb]{0.737,0.353,0.396}{\textbf{#1}}}

\usepackage{framed}

\definecolor{shadecolor}{rgb}{.97, .97, .97}
\definecolor{messagecolor}{rgb}{0, 0, 0}
\definecolor{warningcolor}{rgb}{1, 0, 1}
\definecolor{errorcolor}{rgb}{1, 0, 0}
\newenvironment{knitrout}{}{} % an empty environment to be redefined in TeX

\usepackage{alltt}
\usepackage[sc]{mathpazo}
\usepackage[T1]{fontenc}
\usepackage{geometry}
\geometry{verbose,tmargin=2.5cm,bmargin=2.5cm,lmargin=2.5cm,rmargin=2.5cm}
\setcounter{secnumdepth}{2}
\setcounter{tocdepth}{2}
\usepackage{url}
\usepackage[unicode=true,pdfusetitle,
 bookmarks=true,bookmarksnumbered=true,bookmarksopen=true,bookmarksopenlevel=2,
 breaklinks=false,pdfborder={0 0 1},backref=false,colorlinks=false]
 {hyperref}
\hypersetup{
 pdfstartview={XYZ null null 1}}
\usepackage{breakurl}
\IfFileExists{upquote.sty}{\usepackage{upquote}}{}
\begin{document}

\title{A Minimal Demo of knitr}
\author{Jason A. French}
\maketitle

\section{Methods}
We recruited 200 university undergraduates from an introductory psychology class. Participants were drawn from various genders and ethnic groups across the Chicago area (see Table~\ref{tab:ethnicity})...

\begin{table}[ht]
\centering
\begin{tabular}{rrr}
\hline
 & Female & Male \\
\hline
African American &  22 &  16 \\
Asian American &  26 &  30 \\
Latino &  14 &  30 \\
Pacific Islander &  18 &  14 \\
White &  20 &  10 \\
\hline
\end{tabular}
\caption{Participant Ethnicities}
\label{tab:ethnicity}
\end{table}
\end{document}

Last, after compiling our LaTeX file using TeX Shop, we’re greeted with the final product below:

Summary thus far

The example above used data from R directly in a sentence in the Methods section (i.e., “We recruited 200 university undergraduates from an introductory psychology class.”) and did so using the \Sexpr{} command in the knitr manuscript (i.e., knitr.Rnw). The \Sexpr{} command contained an R expression to calculate the total number participants. This expression was evaluated and converted to LaTeX code when we ran the knit() function on the .Rnw file, which produces a .tex document. The .tex document contained no R code and was therefore ready to be compiled to a PDF using TeX Shop or pdf2latex in Terminal.app.

Forcing knitr to round to 2 decimal places

The default behavior of knitr works well most of the time. However, what if we didn’t have whole numbers in our data table? What if we had percentages that we wanted to round down to 2 digits, as required by many journals? For example, the value \Sexpr{pi} would be evaluated and replaced with 3.141593 in the LaTeX file. One common problem, and part of Yihui’s motivation for replacing Sweave with knitr, is that \Sexpr{} doesn’t automatically round digits.

In Sweave (i.e., knitr’s predecessor), each value of pi would have to be encased in round(pi,2). Thus, we end up with \Sexpr{round(pi,2)}. Yihui fixed this problem by automatically rounding digits, the length of which is set with options(digits=2) in the knitr preamble in your .Rnw document. See below:

1
2
3
4

<<>>=
library(knitr)
options(digits=2)
@

The default rounding behavior of knitr works well until a value contains a 0 after rounding, such as 123.10. Running the expression round(123.10,2) outputs 123.1. In this case, every other value in the manuscript table would be aligned at the decimal place except for the unlucky value - sticking out like a sore thumb. To fix this, you could use sprintf("%.2f", pi) every time you have to call \Sexpr{} in the manuscript - but then what’s the advantage of using knitr? This hack unnecessarily complicates the manuscript and distracts from the writing process.

Modify the default inline_hook for knitr

After seeing a StackOverflow answer by Josh O’Brien, I realized that the default inline_hook function for knitr could be easily modified to use the sprintf() command instead of round(). The minute change will forcibly output all manuscript values to 2 decimal places. Below, we see the default behavior for knitr when processing inline R expressions:

1
2

library(knitr)
knit_hooks$get("inline")

1
2
3
4
5
6

function (x)
{
    if (is.numeric(x))
        x = round(x, getOption("digits"))
    paste(as.character(x), collapse = ", ")
}

Note: My original code for this post used the format() command. Winston Chang pointed out that this could lead to unreliable output and tweaked the code to use sprintf(). The credit for the more efficient function below goes to him. Below, we add out improved inline_hook to the preamble of our knitr document:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

<<>>=
library(knitr)
inline_hook <- function (x) {
  if (is.numeric(x)) {
    # ifelse does a vectorized comparison
    # If integer, print without decimal; otherwise print two places
    res <- ifelse(x == round(x),
      sprintf("%d", x),
      sprintf("%.2f", x)
    )
    paste(res, collapse = ", ")
  }
}
knit_hooks$set(inline = inline_hook)
@

Working Example

Let’s put it all together! The following is a working example of the the suggested knitr inline_hook function, which should give more reliable output by rounding inline values to 2 decimal places.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

\documentclass[12pt]{article}
\begin{document}

<<load, include=FALSE, echo=FALSE>>=
library(knitr)

inline_hook <- function (x) {
  if (is.numeric(x)) {
    # ifelse does a vectorized comparison
    # If integer, print without decimal; otherwise print two places
    res <- ifelse(x == round(x),
      sprintf("%d", x),
      sprintf("%.2f", x)
    )
    paste(res, collapse = ", ")
  }
}

knit_hooks$set(inline = inline_hook)
@

Inline code looks like \Sexpr{123}, \Sexpr{123.4}, \Sexpr{123.45}, \Sexpr{123.456}.

And with vectors: \Sexpr{c(123, 123.4, 123.45, 123.456)}.

Regular output is not affected by the inline hook:

<<>>=
123
123.4
123.45
123.456

c(123, 123.4, 123.45, 123.456)

getOption('digits')
@
\end{document}

Polychoric correlations estimate the correlation between two theorized normal distributions given two ordinal variables. In psychological research, much of our data fits this definition. For example, many survey studies used with introductory psychology pools use Likert scale items. The responses to these items typically range from 1 (Strongly disagree) to 6 (Strongly agree). However, we don’t really think that person’s relationship to the item is actually polytomous. Instead, it’s an imperfect approximation.

Similarly, tetrachorics are special cases of polychoric crrelations when the variable of interest is dichotomous. The participant may have gotten the item either correct (i.e., 1) or incorrect (i.e., 0), but the underlying knowledge that led to the items’ response is probably a continuous distribution.

When you have polytomous rating scales but want to disattenuate the correlations to more accurately estimate the correlation betwen the latent continuous variables, one way of doing this is to use a tetrachoric or polychoric correlation coefficient.

The problem

At the SAPA Project, the majority of our data is polytomous. We ask you the degree to which you like to go to lively parties to estimate your score on latent extraversion. Presently, we use mixed.cor(), which calls a combination of the tetrachoric() and polychoric() functions in the psych package (Revelle, W., 2013).

However, each time we build a new dataset from the website’s SQL server, it takes hours. And that’s if everything goes well. If there’s an error in the code or a bug in a new function, it may take hours to hit the error, wasting your day.

After a bit of profiling, it was revealed that much of our time building the SAPA dataset was used estimating the tetrachoric and polychoric correlation coefficients. When you do this for 250,000+ participants for 10,000+ variables, it takes a long time. So Bill and I thought about how we could speed them up and feel others may benefit from our optimization.

A serious speedup to tetrachoric and polychoric was initiated with the help of Bill Revelle. The increase in speed is roughly 1- (nc-1)² / nc² where nc is the number of categories. Thus, for tetrachorics where nc=2, this is a 75% reduction, whereas for polychorics of 6 item responses this is just a 30% reduction.

Optimizing the existing function

If we call psych:::tetraBinBvn, we see that most of the time spent computer the tetrachorics is using pmvnorm to estimate each of the 4 probabilities of the tetrachoric:

The function says for each row i and each column j, optimize a probability based on the lower and upper bounds of the row cuts and column cuts. But wait! Since the sum of the probabilities in the tetrachoric 2x2 matrix add to 1, why not just use a bit of subtraction to figure it out? That would avoid waiting on pvnorm to compute each cell.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

function (rho, rc, cc)
{
    row.cuts <- c(-Inf, rc, Inf)
    col.cuts <- c(-Inf, cc, Inf)
    P <- matrix(0, 2, 2)
    R <- matrix(c(1, rho, rho, 1), 2, 2)
    for (i in 1:2) {
        for (j in 1:2) {
            P[i, j] <- pmvnorm(lower = c(row.cuts[i], col.cuts[j]),
                upper = c(row.cuts[i + 1], col.cuts[j + 1]),
                corr = R)
        }
    }
    P
}

1
2
3
4
5
6
7
8
9
10
11
12
13
14

function (rho, rc, cc)
{
    row.cuts <- c(-Inf, rc, Inf)
    col.cuts <- c(-Inf, cc, Inf)
    P <- matrix(0, 2, 2)
    R <- matrix(c(1, rho, rho, 1), 2, 2)
    P[1, 1] <- pmvnorm(lower = c(row.cuts[1], col.cuts[1]), upper = c(row.cuts[2],
        col.cuts[2]), corr = R)
    P[1, 2] <- pnorm(rc) - P[1, 1]
    P[2, 1] <- pnorm(cc) - P[1, 1]
    P[2, 2] <- 1 - pnorm(rc) - P[2, 1]
    P
}
<environment: namespace:psych>

This solution resulted in a speed up of tetrachor() by a factor of 3!

Unfortunately, our trick doesn’t work for the polychor() function as well, since there are more degrees of freedom. However, we can eliminate calling pvnorm() for 11 cells in a hypothetical 6x6 matrix as we only need to find the values for 25 and can use subtraction to speed up the estimation of the remaining 11. This results in a less impressive speed up for polychor() that I nonetheless hope will benefit researchers.

Writing a parallelized function

While we were happy with our initial speed increase, it didn’t have a noticeable impact on SAPA’s build time. We then began to explore ways to parallelize our existing code. While we attempted to use various packages (i.e., foreach), we settled on using mcmapply in the existing core parallel library to minimize our dependencies.

As you can see below, this resulted in a speed increase by a factor of N, where N = # of cpus you allocate. Be warned, however, that memory consumption increases dramatically as well.

Item Response Theory can be used to evaluate the effectiveness of exams given to students. One distinguishing feature from other paradigms is that it does not assume that every question is equally difficult (or that the difficulty is tied to what the researcher said). In this way, it is an empirical investigation into the effectiveness of a given exam and can help the researcher 1) eliminate bad or problematic items and 2) judge whether the test was too difficult or the students simply didn’t study.

In the following tutorial, we’ll use R (R Core Team, 2013) along with the psych package (Revelle, W., 2013) to look at a hypothetical exam.

Before we get started, remember that R is a programming language. In the examples below, I perform operations on data using functions like cor and read.csv. We can also save the output as objects using the assignment arrow, <-. It’s a bit different from a point-and-click program like SPSS, but you don’t need to know how to program to analyze exams and questions using IRT!

First, load the the psych package. Next, load the students’ grades into R using read.csv() from the psych package.

1
2
3
4

# Load the psych package
library(psych)
# Read the csv file and save it as grades
grades <- read.csv(file = "~/Downloads/Grades.csv")

Notice that we are using item-level grades, where each row is a given student and each cell is the number of points received on that question. In my example, V1, V2, etc. correspond to exam questions. Your matrix or data frame should look like this:

1

head(grades)

1
2
3
4
5
6
7
8
9
10
11
12
13
14

##   V1 V2 V3 V4 V5 V6 V7 V8 V9 V10 V11 V12 V13 V14 V15 V16 V17 V18 V19 V20
## 1  2  2  2  2  4  2  3  2  4   2   2   3   5   3   6   2   4   4   4   3
## 2  2  0  2  2  0  0  3  0  4   2   0   1   3   0   0   2   2   1   0   0
## 3  2  2  2  2  0  2  3  2  4   0   0   3   5   0   5   2   4   2   4   2
## 4  2  1  2  2  3  0  0  2  4   2   2   3   1   1   2   0   4   1   4   3
## 5  2  2  2  2  4  2  3  2  4   2   2   3   5   3   4   2   4   4   4   3
## 6  1  0  2  2  0  2  0  2  0   2   1   2   5   0   3   2   4   2   4   3
##   V21 V22 V23 V24 V25 V26 V27 Total
## 1   2   2   4   4   6   6   6    91
## 2   0   0   0   3   3   6   6    42
## 3   2   1   2   4   6   5   6    72
## 4   0   2   4   4   0   0   0    49
## 5   2   2   4   4   5   6   6    88
## 6   0   0   4   4   6   4   3    58

Next, compute the polychoric correlations on the raw grades (not including the Total column). By using polychoric correlations, we estimate the normal distribution of latent content knowledge, which can be underestimated if Pearson correlations are instead used on polytomous items (see Lee, Poon & Bentler, 1995).

1
2

# Perform a polychoric correlation on grades and save it as grades.poly
grades.poly <- polychoric(x = grades[, 1:27], polycor = TRUE)

Now that we have the polychoric correlations, we can run irt.fa() on the dataset to see the item difficulties and information.

1
2
3
4

# Using grades.poly, perform an irt and save it as grades.irt
grades.irt <- irt.fa(x = grades.poly, plot = TRUE)
# Plot the output from grades.irt
plot(grades.irt)

Thus, we have some great items that have a lot of information about students of average and low content knowledge (e.g., V24, V17, V18), but not enough to distinguish the high-knowledge students. In redesigning an exam for next semester or year, we might save the best performing questions while trying to rewrite the existing questions or trying new questions.

Next, let’s see what how well the test did overall at distinguishing students:

1

plot.irt(type = "test", grades.irt)

The second plot shows the test performance. We have great reliability for distinguishing who didn’t study (lowers end of our latent trait), but overall the test may have been too easy (opposite of my prediction). It’s important that the test is not too difficult to discourage students, but the graph above suggests that we had very low information at how students that studied were different from eachother. This is again reflected by the histogram plotted in the next section, where high scoring students seem to cluster together.

Rescaling the test

While many students in our hypothetical dataset did very well on the exam, instructors may need to rescale their exam so that the mean grade is an 85% or 87.5%. Using the scale() function (see also rescale() in psych package), we can ensure that the rank-order distribution of the students is preserved (allowing us to distinguish those who studied well from those who didn’t), while scaling the sample distribution to fit in with other classes in your department.

Currently, our scores are in cumulative raw points. Notice that we divide the Total points column by 91 to convert the histogram into grade percentages. Let’s plot a histogram to see the distribution of scores.

1
2
3
4
5

# Load the ggplot2 library
library(ggplot2)
# Plot a histogram
qplot((Total/91)*100, data=grades, geom="histogram",xlab='Raw Grades',
  ylab='# of Students',main='Distribution of student grades')

The distribution has a mean 71.68 percent and a standard deviation of 15.54. Given grade inflation, it may look like your students are doing poorly when in fact the distribution is similiar to other courses being taught. Next, we can rescale the grades, creating a mean of 87.5 and a standard deviation of 7.5. These numbers are arbitrary so use your best judgement.

1
2
3

scaled.grades <- scale(grades$Total) * 7.5 + 87.5
qplot(scaled.grades, xlab='Scaled Grades',
  ylab='# of Students',main='Distribution of student grades')

The second distribution may be preferred, depending on your needs. With the raw distribution, we would have had 45% of the students receiving grades below a C-, assuming a normal distribution (for the curious, R can calculate these probabilites using the pnorm function: pnorm(q=70,mean=71.68,sd=15.54)). Now, 0.9% of students would fall below the 70% cutoff. Again, my mean and standard deviation chosen in the above example are arbitrary.

References

• Lee, S. Y., Poon, W. Y., & Bentler, P. M. (1995). A two‐stage estimation of structural equation models with continuous and polytomous variables. British Journal of Mathematical and Statistical Psychology, 48(2), 339-358.

• R Core Team (2013). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. http://www.R-project.org/.

• Revelle, W. (2013). psych: Procedures for Personality and Psychological Research. Northwestern University, Evanston, Illinois, USA. http://CRAN.R-project.org/package=psych. Version = 1.3.10.

Needless to say, the marriage of statistics with documents makes writing up APA-style reports a bit easier, especially with Brian Beitzel’s amazing apa6 class for LaTeX.

Making Sweave Available Systemwide

However, Sweave doesn’t always work correctly. One common complaint that you’ll get after Sweaving a file is Sweave.sty not found!. While Sweave.sty is a LaTeX package, it doesn’t live with the rest of the LaTeX packages because it’s installed using R. Many people try to solve this by copying and pasting Sweave.sty into every document directory, but I’m sharing a better way below.

Using Terminal.app:

1

cd /usr/local/texlive/texmf-local/tex/latex

1

sudo ln -s /Library/Frameworks/R.framework/Resources/share/texmf/ Sweave

1

sudo mktexlsr

If using mktexlsr results in a command not found error, the TeX Live distribution probably isn’t in your $PATH, but you can hunt for the program anyway. For example, if you’re using MacTeX 2013, the program will be found in a directory similar to this:

1

sudo /usr/local/texlive/2013/bin/x86_64-darwin/mktexlsr

Using Sweave in TeXShop

When you’re getting started with LaTeX, many Mac users prefer the bundled editor, TeXShop. Cameron Bracken gives us a helpful piece of code that allows easy Sweaving straight from TeXShop. TeXShop uses various engines that allow it to render LaTeX. Using a bit of BASH scripting, we can write our own Sweave engine and make it available right within TeXShop. I have adapted Cameron’s original engine to accomodate Bibtex citations (see here).

Using a text editor, paste in the following syntax and save the file as Sweave.engine:

1
2
3
4
5
6
7
8

#!/bin/env bash
export PATH=$PATH:/usr/texbin:/usr/local/bin
R CMD Sweave "$1"
pdflatex "${1%.*}"
bibtex "${1%.*}.aux"
pdflatex "${1%.*}"
# If you run pdflatex again you get citations
pdflatex "${1%.*}"

Next, in Terminal.app, move the file to the TeXShop engines folder:

1
2

mv Sweave.engine ~/Library/TeXShop/Engines/
chmod +x ~/Library/TeXShop/Engines/Sweave.engine

Now restart TeXShop if it’s running and you should see Sweave as an available option!

These instructions are organized by system type.

Debian-based Distributions

Ubuntu

Full installation instructions for Ubuntu can be found here. Luckily, CRAN mirrors have compiled binaries of R which can be installed using the apt-get package manager. To accomplish this, we’ll first add the CRAN repo for Ubuntu packages to /etc/apt/sources.list. If you prefer to manually edit the sources.list file, you can do so by issuing the following in the terminal:

1

sudo nano /etc/apt/sources.list

1
2
3
4
5
6
7
8
9
10
11

# Grabs your version of Ubuntu as a BASH variable
CODENAME=`grep CODENAME /etc/lsb-release | cut -c 18-`

# Appends the CRAN repository to your sources.list file 
sudo sh -c 'echo "deb http://cran.rstudio.com/bin/linux/ubuntu $CODENAME" >> /etc/apt/sources.list'

# Adds the CRAN GPG key, which is used to sign the R packages for security.
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys E084DAB9

sudo apt-get update
sudo apt-get install r-base r-dev

Debian

The instructions for installing R in Debian are similar to Ubuntu. Regarding ‘stable’ versions of Debian, the CRAN README file for Debian points out:

After a release of Debian “stable”, no new packages get added by Debian to keep the release as ‘stable’ as possible. This implies that the R release contained in the official Debian release will become outdated as time passes.

Thus, we’ll append the CRAN repository to the Debian list to update the available R version, just like we did for Ubuntu:

1
2
3
4
5
6
7

# Appends the CRAN repository to your sources.list file 
sudo sh -c 'echo "deb http://cran.rstudio.com/bin/linux/debian lenny-cran/" >> /etc/apt/sources.list'

# Adds the CRAN GPG key, which is used to sign the R packages for security.
sudo apt-key adv --keyserver subkeys.pgp.net --recv-key 381BA480
sudo apt-get update
sudo apt-get install r-base r-base-dev

Finally, you can search for additional R packages in terminal using apt-cache:

1

apt-cache search ^r-.*

Installing RStudio

Rstudio is a cross-platform user interface for R. The RStudio package is compiled for both Debian and Ubuntu distributions. Therefore, the installation instructions are the same. Copy the link for the latest RStudio package from http://www.rstudio.com/ide/download/desktop (e.g., http://download1.rstudio.org/rstudio-0.97.320-amd64.deb ).

1

sudo apt-get install http://download1.rstudio.org/rstudio-0.97.320-amd64.deb

RedHat-based Distributions

RedHat EL6 (or CentOS 6+)

In order to get R running on RHEL 6, we’ll need to add an additional repository that allows us to install the new packages, EPEL. Extra Packages for Enterprise Linux (or EPEL) is a Fedora Special Interest Group that creates, maintains, and manages a high quality set of additional packages for Enterprise Linux, including, but not limited to, Red Hat Enterprise Linux (RHEL), CentOS and Scientific Linux (SL).

1
2
3
4
5
6
7
8
9

# For El5 or CentOS 5
su -c 'rpm -Uvh http://download.fedoraproject.org/pub/epel/5/i386/epel-release-5-4.noarch.rpm'
sudo yum update
sudo yum install R

# For El6 or CentOS 6
su -c 'rpm -Uvh http://download.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm'
sudo yum update
sudo yum install R

Finally, you can search for additional R packages in terminal using yum:

1

yum list R-\*

Fedora

For Fedora, life is much easier. Current versions of Fedora have an up-to-date build of R in their repositories.

1
2

sudo yum update
sudo yum install R

Installing RStudio

You’ll want to copy the appropriate link for your system from the RStudio Desktop Download page. In my case, it was http://download1.rstudio.org/rstudio-0.97.320-x86_64.rpm.

1

sudo yum install http://download1.rstudio.org/rstudio-0.97.320-x86_64.rpm

Dealing with PPC-based Systems

As our lab has a lot of old iMacs that are still quite useful, I’ve recently dealt with the best way to support R on these machines. These machines are still quite useful, particulary when their maximum potential RAM is installed. However, Apple stopped supporting the machines after OS 10.5.8 (see here). While the nightly packages distributed do install on PPC machines with OS X, they lack the R Framework compiled for PPC, meaning that it’s a useless installation. This means that a user stuck on OS X 10.5.8 is tied to R 2.10, a very old R distribution that’s incompatible with many existing packages.

Any solutions?

My first solution was to compile R from source using MacPorts, a package manager similar in concept to yum or apt-get. While successful, it takes a long, long, time to build R and its necessary dependencies on a 1.8 Ghz G5 processor. From a system administrator’s perspective, this also is the least parsimonious solution possible, since each machine has to be updated with each new release of R.

Thanks to RStudio Server, each machine doesn’t need to have R installed, as it can be run off a more powerful server and accessed using a reasonably up-to-date browser. I was able to install R and RStudio on our RedHat EL6 server easily. The trick was to make this as seemless as possible from the user’s perspective. To accomplish this, I saved the Bookmark to the Desktop.

Next, I downloaded a large R icon using Google Images and edited the Bookmark’s icon to appear as if it were R. To do this, just copy the R icon from within Preview, select the icon of the Bookmark by right-clicking and selecting Get Info, and pasting using Command+V.

Finally, this was dragged to the OS X dock, appearing just as if it were R on the local machine, but without all the hassle and slow load times on PPC.

It is worth noting that this solution is only important for users tied to OS X 10.5.8. I’ve had great success with using Fedora’s PPC build, which has packages compiled for PPC already. However, as Linux is intimidating for many users, I chose to install RStudio Server on our lab server.

Frequently Asked Questions

1. I get an error that says I’m not in the sudoers file: This means that you don’t have access to install software on your machine. Talk to your system administrator.

2. I don’t know if I’m running Ubuntu or Fedora. How do I know which instructions to use? Jokes aside, run lsb_release -irc in your terminal.

While browsing a friend’s .bashrc on Github, I realized I could use a simple Bash function to speed things up. Copy and paste the following into Terminal:

1
2
3
4
5
6

echo "function Rsscc() {
    ssh  -c arcfour,blowfish-cbc \
        -XC netid@hardin.it.northwestern.edu rstudio
    wait $1
    exit 0
}" >> ~/.profile

After you restart Terminal.app, you can launch RStudio remotely by typing Rsscc, or whatever you renamed my function to. In principle, you could also create a simple menu for choosing among multiple servers or programs using a bit of read and case.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

task_menu () {
cat << EOF
$(tput setaf 5)Remote Login$(tput sgr0)
$(tput setaf 5)============$(tput sgr0)

Please choose one of the following:

1) RStudio
2) Stata

EOF
  read -r choice
  case "$choice" in
      1) task="rstudio" ;;
      2) task="xstata" ;;
      *) echo "Please choose a number!" && task_menu ;;
  esac
fi
ssh -c arcfour,blowfish-cbc \
    -XC netid@hardin.it.northwestern.edu $task
    wait $1
    exit 0
}

Note: This works best if you’re using an up-to-date version of X11, such as XQuartz and are accessing the SSCC using Ethernet.

Jason Bryer has recently developed an R package published to Github for downloading data from Qualtrics within R using the Qualtrics API (see his Github repo). Using this package, you can integrate your Qualtrics data with other experimental data collected in the lab and, by running an Rscript as a cronjob, get daily updates for your analyses in R. I’ll demonstrate the use of this package below.

Installing the Qualtrics Package from Github

1
2
3
4
5
6
7

#!/usr/bin/env Rscript
# Authors:  Jason A. French
# Email:    frenchja@u.northwestern.edu
install.packages('devtools',dependencies=TRUE)
# install.packages('RTools') # May be needed for some Windows versions.
devtools:::install_github(repo='qualtrics',username='jbryer')
require(qualtrics)

If you wanted to distribute the Rscript to colleagues with different setups, you would use something like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

#!/usr/bin/env Rscript
# Authors:  Jason A. French
# Email:    frenchja@u.northwestern.edu
# Check for devtools
if (!require(devtools)) {
    warning('devtools not found. Attempting to install!')
    install.packages('devtools',dependencies=TRUE)
    require(devtools,character.only=TRUE)
}

# Check Qualtrics Package from Github
if (!require(qualtrics)) {
  warning('Qualtrics package not found')
  install_github(repo='qualtrics',username='jbryer')
  require(qualtrics)
}

Pulling Data from Qualtrics

First, you’ll need to find the Survey ID of the associated study on Qualtrics. Access your Account Settings and click on Qualtrics IDs. From here, copy the entire ID into your R code (e.g., SV_blahblah).

Next, load the qualtrics package in R and pull the data using getSurveyResults(). If you receive an error in R regarding API permissions, you may need to email support@qualtrics.com and request API access. The R function accesses your survey using XML.

1
2
3
4

# Get Qualtrics Survey Data
qualtrics.data <- getSurveyResults(username=qualtrics.user,
                      password=qualtrics.pass,
                      surveyid='SV_blahblah')

However, readers have pointed out that the resulting data doesn’t have your variable names. Alternatively, Trevor Kvaran points out that you can modify the getSurveyResults function to export the variable names by appending “&ExportTags=1” to the url:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

getSurveyResults <- function (username, password, surveyid, truncNames = 20, startDate = NULL, endDate = NULL)
{
    url = paste("http://eu.qualtrics.com/Server...", username,
                "&Password=", password,
                "&SurveyID=", surveyid,
                "&ExportTags=1",
                "&Format=CSV",
                ifelse(is.null(startDate), "", paste("&StartDate=", startDate, sep = "")), ifelse(is.null(endDate), "", paste("&EndDate=", endDate, sep = "")), sep = "")

    t = read.csv(url)
    t<-t[-1,]
    t$X = NULL
    n = strsplit(names(t), "....", fixed = TRUE)
    for (i in 1:ncol(t)) {
        names(t)[i] = n[[i]][length(n[[i]])]
        if (nchar(names(t)[i]) > truncNames) {
            names(t)[i] = substr(names(t)[i], 1, truncNames)
        }
    }
    t
}

Again, if you are distributing the analysis to collaborators, you may want to use a setup like below, which reads the user and password securely. If you want to automate this, you’ll obviously need hardcode your username and password. At this time, I don’t believe R has a hashing mechanism for passwords like Python, meaning you’ll need to have a plaintext password.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

# Get Qualtrics Survey Data
# Not sure this will work in R.app.  Only in Terminal.app
get_password <- function() {
  cat("Qualtrics Password: ")
  system("stty -echo")
  a <- readline()
  system("stty echo")
  cat("\n")
  return(a)
}

qualtrics.user <- readline(prompt='Enter your Qualtrics username')
qualtrics.pass <- get_password()
qualtrics.data <- getSurvey(username=qualtrics.user,
                      password=qualtrics.pass,
                      surveyid='SV_blahblah')

Outputting Data using xtable

Now that we have our data pulled from Qualtrics, we can analyze it as if it were a regular R data.frame. If you know your specific survey variables of interest, you’ll want to modify the code using qualtrics.data[,c('Var1','Var2)].

1
2
3
4
5
6
7
8
9
10

require(psych)
require(xtable)
qualtrics.describe <- describe(qualtrics.data)
qual.table <- xtable(qualtrics.describe)
print.xtable(qual.table,type='html',file='~/Desktop/Qualtrics.html')

# Or...
qual.lm <- lm(DV ~ IV1*IV2, data=qualtrics.data)
lm.table <- xtable(qual.lm)
print.xtable(lm.table,type="html",file='~/Desktop/Qualtrics.html',append=TRUE)

Running an Rscript

In order for your script to be run daily, it needs to be a) marked as executable (see here) and b) added to crontab. First, mark it as executable:

1

chmod +x ~/Qualtrics.R

Make sure that #!/usr/bin/env Rscript is at the top of your script!

Next, edit the system’s crontab and add the script.

1
2
3

sudo nano /etc/crontab

0 1 * * * ~/Qualtrics.R

Consider the following. When running an ANOVA, the test accounts for three sources of variance: 1) the fixed effect of the condition, 2) the ability of the participants, and 3) the random error, as data = model + error. Plotting the repeated-measures without taking the different sources of variance into consideration would result in overlapping error bars that include between-subject variability, confusing the presentation’s audience. While the ANOVA partials out the differences between the participants and allow you to assess the effect of the repeated-measure, computing a regular confidence interval by multiplying the standard error and the F-statistic doesn’t work in this way.

Winston Chang has developed a set of R functions based on Morey (2008) and Cousineau (2005) on his wiki that help deal with this problem, where the sample variance is computed for the normalized data, and then multiplied by the sample variances in each condition by M(M-1), where M is the number of within-subject conditions.

See his wiki here for more info.

References

Morey, R. D. (2008). Confidence Intervals from Normalized Data:   A correction to Cousineau (2005).

Cousineau,D. (2005). Confidence intervals in within‐subject designs:A simpler solution to Loftus and Masson’s method.

Loftus, G.R., & Masson, M.E.J. (1994). Using confidence intervals in within‐subject designs.

In order to make test items or stimuli easier to understand, researchers occasionally organize examples in a table or figure. Using the standard \table command in LaTeX, it’s possible to include figures in an individual table cell without breaking the APA6.cls package. For example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42

\begin{table}
  [ht] \caption{Stimuli Category Explanations} \label{tab:stimuli}
  \begin{tabular}
      {lllll} \hline Category & Familiar Organism & Growth Model & Metamorphosis & Figure \\
      \hline \textbf{FDM} & Yes & Dramatic & Yes &
      \includegraphics[width=1in]{"Figure Stimuli"/Fam-Meta-Dramatic} \\
      FDN & Yes & Dramatic & No &
      \includegraphics[width=1in]{"Figure Stimuli"/Fam-Dramatic} \\
      FGM & Yes & Growth & Yes &
      \includegraphics[width=1in]{"Figure Stimuli"/Fam-Meta-Growth} \\
      \textbf{FGN} & Yes & Growth & No &
      \includegraphics[width=1in]{"Figure Stimuli"/Fam-Growth} \\
      FIM & Yes & Identical & Yes &
      \includegraphics[width=1in]{"Figure Stimuli"/Fam-Meta-ID} \\
      FIN & Yes & Identical & No &
      \includegraphics[width=1in]{"Figure Stimuli"/Fam-Identical} \\
      FSM & Yes & Species Change & Yes &
      \includegraphics[width=1in]{"Figure Stimuli"/Fam-Meta-Species} \\
      FSN & Yes & Species Change & No &
      \includegraphics[width=1in]{"Figure Stimuli"/Fam-Species} \\
      \textbf{UDM} & No & Dramatic & Yes &
      \includegraphics[width=1in]{"Figure Stimuli"/Un2-Meta-Dramatic} \\
      UDN & No & Dramatic & No &
      \includegraphics[width=1in]{"Figure Stimuli"/Un-Dramatic} \\
      UGM & No & Growth & Yes &
      \includegraphics[width=1in]{"Figure Stimuli"/Un2-Meta-Growth} \\
      \textbf{UGN} & No & Growth & No &
      \includegraphics[width=1in]{"Figure Stimuli"/Un-Growth} \\
      UIM & No & Identical & Yes &
      \includegraphics[width=1in]{"Figure Stimuli"/Un2-Meta-ID} \\
      UIN & No & Identical & No &
      \includegraphics[width=1in]{"Figure Stimuli"/Un-Identical} \\
      USM & No & Species Change & Yes &
      \includegraphics[width=1in]{"Figure Stimuli"/Un2-Meta-Species} \\
      USN & No & Species Change & No &
      \includegraphics[width=1in]{"Figure Stimuli"/Un-Species} \\
      USM & No & Species Change & Yes &
      \includegraphics[width=1in]{"Figure Stimuli"/Un2-Meta-Species} \\
      \hline
  \end{tabular}
  \tabfnt{Note: Bold categories are biologically correct growth models.}
\end{table}

Center-Aligned Figures

However, the above code vertically aligned my images according to their bottom-edge, producing an awkward looking table. Instead, we want the figures to be vertically centered. A Google search revealed the LaTeX Wikibook, which suggests a few methods to force figures to vertically align according to their center. Below, I surround each \includegraphics{} command with the \parbox{} command, which centers it along 1 unit of measurement, set to 12 pts. in my apa6 class options.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42

\begin{table}
  [ht] \caption{Stimuli Category Explanations} \label{tab:stimuli}
  \begin{tabular}
      {lllll} \hline Stimuli Category & Familiar Organism & Growth Model & Metamorphosis & Figure \\
      \hline \textbf{FDM} & Yes & Dramatic & Yes & \parbox[c]{1em}{
      \includegraphics[width=1in]{"Figure Stimuli"/Fam-Meta-Dramatic}} \\
      FDN & Yes & Dramatic & No & \parbox[c]{1em}{
      \includegraphics[width=1in]{"Figure Stimuli"/Fam-Dramatic}} \\
      FGM & Yes & Growth & Yes & \parbox[c]{1em}{
      \includegraphics[width=1in]{"Figure Stimuli"/Fam-Meta-Growth}} \\
      \textbf{FGN} & Yes & Growth & No & \parbox[c]{1em}{
      \includegraphics[width=1in]{"Figure Stimuli"/Fam-Growth}} \\
      FIM & Yes & Identical & Yes & \parbox[c]{1em}{
      \includegraphics[width=1in]{"Figure Stimuli"/Fam-Meta-ID}} \\
      FIN & Yes & Identical & No & \parbox[c]{1em}{
      \includegraphics[width=1in]{"Figure Stimuli"/Fam-Identical}} \\
      FSM & Yes & Species Change & Yes & \parbox[c]{1em}{
      \includegraphics[width=1in]{"Figure Stimuli"/Fam-Meta-Species}} \\
      FSN & Yes & Species Change & No & \parbox[c]{1em}{
      \includegraphics[width=1in]{"Figure Stimuli"/Fam-Species}} \\
      \textbf{UDM} & No & Dramatic & Yes & \parbox[c]{1em}{
      \includegraphics[width=1in]{"Figure Stimuli"/Un2-Meta-Dramatic}} \\
      UDN & No & Dramatic & No & \parbox[c]{1em}{
      \includegraphics[width=1in]{"Figure Stimuli"/Un-Dramatic}} \\
      UGM & No & Growth & Yes & \parbox[c]{1em}{
      \includegraphics[width=1in]{"Figure Stimuli"/Un2-Meta-Growth}} \\
      \textbf{UGN} & No & Growth & No & \parbox[c]{1em}{
      \includegraphics[width=1in]{"Figure Stimuli"/Un-Growth}} \\
      UIM & No & Identical & Yes & \parbox[c]{1em}{
      \includegraphics[width=1in]{"Figure Stimuli"/Un2-Meta-ID}} \\
      UIN & No & Identical & No & \parbox[c]{1em}{
      \includegraphics[width=1in]{"Figure Stimuli"/Un-Identical}} \\
      USM & No & Species Change & Yes & \parbox[c]{1em}{
      \includegraphics[width=1in]{"Figure Stimuli"/Un2-Meta-Species}} \\
      USN & No & Species Change & No & \parbox[c]{1em}{
      \includegraphics[width=1in]{"Figure Stimuli"/Un-Species}} \\
      USM & No & Species Change & Yes & \parbox[c]{1em}{
      \includegraphics[width=1in]{"Figure Stimuli"/Un2-Meta-Species}} \\
      \hline
  \end{tabular}
  \tabfnt{Note: Bold categories are biologically correct growth models.}
\end{table}

Output:

By using \parbox, figures are now vertically aligned with text cells. However, with the addition of figures the table is too long and we must span the table across 2-pages. To solve this, split the information across two tables. In this case, I can split by the stimuli category.

Alternatively, Brian Beitzel also pointed out that we can invoke longtable as a class option in APA6.cls, which allows tables to span multiple pages.