Alex Rivera | Logout

Strategies for repeating large chunk of analysis

Asked 2011-06-22T08:47:38.180
22

I find myself in the position of having completed a large chunk of analysis and now need to repeat the analysis with slightly different input assumptions.

The analysis, in this case, involves cluster analysis, plotting several graphs, and exporting cluster ids and other variables of interest. The key point is that it is an extensive analysis, and needs to be repeated and compared only twice.

I considered:

  • Creating a function. This isn't ideal, because then I have to modify my code to know whether I am evaluating in the function or parent environments. This additional effort seems excessive, makes it harder to debug and may introduce side-effects.
  • Wrap it in a for-loop. Again, not ideal, because then I have to create indexing variables, which can also introduce side-effects.
  • Creating some pre-amble code, wrapping the analysis in a separate file and source it. This works, but seems very ugly and sub-optimal.

The objective of the analysis is to finish with a set of objects (in a list, or in separate output files) that I can analyse further for differences.

What is a good strategy for dealing with this type of problem?

r
Edit
Report

2 Answers

3

I like to work with combination of a little shell script, a pdf cropping program and Sweave in those cases. That gives you back nice reports and encourages you to source. Typically I work with several files, almost like creating a package (at least I think it feels like that :) . I have a separate file for the data juggling and separate files for different types of analysis, such as descriptiveStats.R, regressions.R for example.

btw here's my little shell script, 

 #!/bin/sh
 R CMD Sweave docSweave.Rnw
 for file in `ls pdfs`;
 do pdfcrop  pdfs/"$file" pdfs/"$file"
 done
 pdflatex docSweave.tex
 open docSweave.pdf

The Sweave file typically sources the R files mentioned above when needed. I am not sure whether that's what you looking for, but that's my strategy so far. I at least I believe creating transparent, reproducible reports is what helps to follow at least A strategy.

answered 2011-06-29T14:53:56.133
1

I tend to push such results into a global list. I use Common Lisp but then R isn't so different.

answered 2011-07-02T21:44:50.063

Your Answer