####################################################################################################

Electronic appendix to the article:

On Fishing for Significance and Statisticians Degree of Freedom 
in the Era of Big Molecular Data

Anne-Laure Boulesteix* (1), Roman Hornung (1), Willi Sauerbrei (2)

*e-mail: boulesteix@ibe.med.uni-muenchen.de

This article is a chapter (Chapter 6, pp. 155-10) in the book:

Berechenbarkeit der Welt? Philosophie und Wissenschaft im Zeitalter von Big Data
Year: 2017
Editors:  Wolfgang Pietsch, Jrg Wernecke, Maxmilian Ott


Author affiliations:

(1) Department of Medical Informatics, Biometry and Epidemiology,
  University of Munich, Marchioninistr. 15, D-81377, Munich, Germany

(2) Department of Medical Biometry and Statistics,
  Albert-Ludwigs-University Freiburg, Stefan-Meier-Str. 26, D-79104, Munich, Germany

####################################################################################################


Prerequisites:
##############

- MPI environment

- Statistical analysis programme R



General information:
####################

- The folder "FishingPaper" this README is contained in has to be put into the 
  R working directory.
  We use paths of the form "./FishingPaper/...".

  Note: Evaluating the results (see below) can also be performed under Windows.
  For the reproduction of the whole analysis - which is
  performed in parallel - Linux is required however.

- The following subfolders are found in "FishingPaper":

  - "Results": the results of the analysis in the form of Rda files
    and the figure that is produced in the evaluation and shown
    in our chapter;
    "RealdataResults.Rda" contains the results obtained for n = 200; 
    "RealdataResultsSubset.Rda" contains the results obtained for n = 50

  - "Functions": R script "SnowfallFunctions.R" that contains functions used in the 
    analysis

  - "JobScripts": R scripts which performs the analysis (MPI environment required);
    "RealdataStudy.R" performs the analysis for n = 200; "RealdataStudySubset.R"
    performs the analysis for n = 50

  - "Datasetfiles": this folder contains the R script "DownloadAndPreprocessData.R"
     which can be used to automatically download and preprocess the dataset used 
     in the analysis; the raw and the processed data will be stored in this
     folder when executing this R script.



Evaluation of the results:
##########################

- For the evaluation of the results it is not necessary to re-perform the analysis:
  The R script "Evaluation.R" contained in "FishingPaper" can be used to reproduce the
  figure shown in our chapter based on the raw results contained in subfolder "Results"
  in the form of Rda files.



Reproducing the results:
########################

- All R-scripts to perform the large data study are contained in the 
  folder "JobScripts".
