Psychology 543
Research Design and Analysis

2:30 - 4:50 Tuesdays | 3:00 - 4:50 Thursdays
2019 Behavioral Sciences Building

Instructor: R. Chris Fraley
Office: 1050 "A" BSB |

TA: Josh Hemmerich

Course Description

The content of the course will be organized into three sections. We will begin by discussing ways in which mathematics and statistics can be used to quantify psychological phenomena. We'll review some important ideas regarding psychological measurement, ways of quantifying the properties of common statistical distributions, and ways of quantifying relationships between variables. Next, we'll discuss the role of mathematics and statistics in theory development. One of the points I'll emphasize repeatedly throughout the course is that statistics is not the same thing as "data analysis." Statistics and mathematics should be incorporated into the theoretical stage of the scientific process, not just the data analysis stage. We'll discuss some ways of doing this, focusing primarily on the use of linear equations to model the assumed causal relationships between variables. We'll review methods for constructing these models, and various techniques for estimating their parameters. Finally, we'll discuss ways in which mathematics and statistics can be used to evaluate scientific models. We'll focus on three sources of model misfit: (a) errors in measurement, (b) sampling errors, and (c) fundamental problems in the model per se. For each of these issues, we'll review techniques that are used to handle or minimize these problems (e.g., estimating reliability of scores, designing research in order to minimize the standard error of estimates). Finally, we'll review a set of tools commonly used in psychology, null hypothesis significance tests, designed to handle the problem of sampling error. You will need to be aware of "significance tests" because they are widely used. However, these classic techniques are hotly debated among quantitative psychologists in contemporary psychology, and you will need to be aware of their limitations. We'll discuss both their uses and misuses, and focus on alternative methods for dealing with the problem of sampling error (e.g., parameter estimates, effect sizes, confidence intervals).

Historically, Psych 543 is known as the "ANOVA course" in our department because it is typically taught with a strong emphasis on experimental design. As someone who does not conduct a lot of randomized experiments, I tend to emphasize the modeling of psychological phenomena the way they occur "naturally" (i.e., in their full confounded glory). We will briefly review ANOVA, but you need to know in advance that ANOVA will only constitute a small piece of this course. If you plan to focus on experimental research in your graduate career, I encourage you to take an additional class in ANOVA during your second year.

I have six major objectives for this course:

The first is to provide you with a deep, intuitive understanding of some fundamental concepts in statistics, as they apply to the study of psychological phenomena. We obviously will not have the time (or interest) to review every kind of statistical technique that is used by psychologists. Therefore, I plan to make sure you have a strong handle on the basic mathematical and statistical concepts that are common to many applications.

My second objective is to make sure you can use SPSS to conduct some common statistical analyses. Every two weeks, you will be given a new data set and a set of analytic objectives. You will be required to analyze those data in SPSS, and, as explained below, write both a methods and results section. Midway through the course, I will only provide you with the data; you will need to determine on your own how to analyze the data and summarize them appropriately.

Third, in an attempt to help you "think outside the box", I intend to teach you how to program statistical routines in the S-Plus programming environment. Unless you know a little bit about statistical programming, you will never have the freedom to think creatively about mathematics and statistics, and you'll be constrained to run only the analyses that are available in popular statistical packages, such as SPSS. Every other week you will have to complete a programming exercise for S-Plus (e.g., simulating sampling distributions, bootstrapping confidence intervals). Early in the semester I'll spend time explicitly teaching you how to use S-Plus, and I'll walk you through some of the assignments. Later in the semester I'll simply give you a problem and expect you to solve it on your own by using S-Plus.

Fourth, I want you to become experienced in thinking critically about research design. Unfortunately, many studies that are conducted in psychology are not designed carefully enough to answer the basic questions they set out to answer. By providing you with examples to study and critique--and by encouraging you to think critically about your own research ideas--I hope to provide you with skills that will benefit your research in the long run.

My fifth goal is to make you aware of some basic controversies in the application of statistical tools to psychological science. Students often learn about statistics as if they are learning a set of recipes for data analysis, or worse, a set of facts. Many of the most commonly used statistical tools in psychology have been hotly contested since their inception. In order to be an educated consumer and user of statistical techniques, you need to understand these debates and exactly what certain tools are and are not capable of achieving.

Finally, I plan to help you learn how to write professional methods and results sections. Every two weeks you will be asked to write a methods and results section for a hypothetical set of data. I will grade these fairly rigorously, attending not only to the basic organization of your ideas, but the quality and clarity with which you convey them. Although my primary goal is to help you become great a scientist, my secondary goal is to help you become a great writer.

Hays, W. L. (1994). Statistics (5th ed.). Fort Worth: Harcourt College Publishers.

This text is currently being revised, and the new edition will not be available for another year or two. In the meantime, used copies of the most recent edition are available via many outlets on-line (e.g.,'s used book sellers). I encourage you to order a copy and read it at your own pace. I will not be assigning specific chapters from this text, but you will need a fairly comprehensive statistics text on your reference shelf if you plan to pursue a PhD in psychological science. [link to Hays text on]

Salsburg, D. (2001). The lady tasting tea: How statistics revolutionized science in the twentieth century. NY: W. H. Freeman and co.

This book provides a wonderful overview of the history of statistics, with special attention to the major players and the key controversies. Why am I assigning this book? Statistics courses are often taught in a passion-free,
recipe-like manner. This way of characterizing the development of modern statistics overlooks the fact that statisticians have been--and are--quite divided over some of the concepts that psychologists accept as "facts." This book will help make you aware of these debates, and, I hope, facilitate your ability to think critically about "statistical conventions." [link to Salsburg book on]

Krause, A., & Olson, M. (2002). The basics of S-Plus. NY: Springer-Verlag.

This book is recommended (not required) if you would like a useful text on programming in S-Plus. [link to Krause-Olson book on]

Grades will be based on your class participation, the quality of your assignments (S-Plus exercises, SPSS exercises, and Methods/Results exercises), quizzes, and exams.


Adabel Lee
Elgiz Bal
Zia Lakdawalla
Aarati Kasturirangan
Sandhya Krishnan
Ben Jee
Katherine Bailey
Aliza Silver
Dana Rusch
Daniel Evatt

Assignments, datasets, notes, and updates will be listed here throughout the semester. Please treat this webpage as your primary syllabus.


August 29, 2002
Next Tuesday, Sept 3, we will begin class at 3:00 instead of 2:30.

August 29, 2002
Here are some links to some useful UIC resources:

Electronic journals
UIC library | catalog (for book and journal call numbers and locations)
UIC web mail
UIC psychology department

September 12, 2002
Next Tuesday we will discuss the article that was distributed in class this past Tuesday. Be prepared to provide a critical analysis of the research questions, design, and analyses. Also, please turn in a one-page summary of your critique. Please proofread your writing; I will be attending to the quality of your writing.

September 12, 2002
As the semester progresses, you'll need to rely on the computers in 2019 in order to complete assignments, practice, etc. I plan to reserve the classroom for these purposes for the following times:

Mondays : after 5
Tuesdays : after our class
Wednesdays : after 4
Thursdays : after our class

If you are using the room during one of these times, and if you're the last one to leave, please be sure the lights are out and that the door is closed.

October 2, 2002
Next Tuesday (Oct 8) we will discuss the article that was distributed in class this past Tuesday. Be prepared to provide a critical analysis of the research questions, design, and analyses. Also, please turn in a one-page summary of your critique. Please proofread your writing; I will be attending to the quality of your writing.

October 9, 2002
Next Tue
sday (Oct 15) we will have a quiz in class. Please make sure you're comfortable with all the material that we've dicussed in class up to this point.

Lecture Overheads
Overheads and (some) notes from class discussions will be available here. You may click on the lecture title to view an HTML version of the overheads. Alternatively, you may right-click on the PowerPoint option and select "save target" to save the original PowerPoint files.


Lecture 1 | August 29, 2002
The Use of Mathematics and Statistics to Understand the Behavior of Natural Systems
PowerPoint version
S-Plus scripts: orbits2.SSC [right-click to save to disk]

Lecture 2 | September 3, 2002
Quantification and Scales of Measurement
PowerPoint version
Introduction to S-Plus

Lecture 3 | September 5, 2002
Visually Summarizing Score Distributions
PowerPoint version
Creating a web page | Using FTP [powerpoint file]| Making a web page from scratch [PDF file]

Lecture 4 | September 9, 2002
Quantitative Summaries of Score Distributions: Central Tendency
PowerPoint version
Dreamweaver practice & creating web pages | Download NsKit (with WS FTP) here

Lecture 5| September 12, 2002
Quantitative Summaries of Score Distributions: Spread and Dispersion
PowerPoint version

Lecture 6| September 17, 2002
Comparing Scores for Different Variables: The Use of Standardized Scores and Standardized Metrics
PowerPoint version

September 19, 2002
Deriving Quantitative Indices of Association: Spearman's Rank-Order Correlation

September 24, 2002
Deriving Quantitative Indices of Association: Spearman's Rank-Order Correlation [continued discussion]
Lecture 7 | Quantifying the Association Between Two Variables: Covariance
PowerPoint version

September 26, 2002
Deriving Quantitative Indices of Association: Correlation
Lecture 8| Quantifying the Association Between Two Variables: Correlation
PowerPoint version

[right-click; select "save target as", change file type to "all files"; save as dataset1.sav; open file in SPSS]

October 1, 2002
Lecture 9 | Mathematical Modeling of Theoretical Relationships: Function Forms and their Specification
PowerPoint version

assignment due: derivation of the correlation using the squared difference of z-scores rather than the products of z-scores

October 3, 2002
Lecture 10 | Least Squares: Estimating the Parameters of Simple Linear Models
PowerPoint version

October 8, 2002
Lecture 11 | Evaulating Model Fit: R-Squared
PowerPoint version

October 10, 2002
Lecture 12 | Using R-Squared to Compare the Fit of Alternative Models
PowerPoint version

splustutorial1.SSC [right-click; select "save target as", change file type to "all files"; save as splustutorial1.ssc; open file in S-Plus]

October 15, 2002
Lecture 13 | Confounds and Complex Causal Structures: Tracing Rules and Methodological and Statistical Control
PowerPoint version

dataset2.sav | dataset3.sav [right-click; select "save target as", change file type to "all files"; save as dataset2.sav and dataset3.sav, respectively; open file in SPSS]

October 22, 2002
Lecture 14 | Additive vs. Nonadditive Effects: Interactions with Binary and Continuous Predictors
PowerPoint version

October 29, 2002
Lecture 15 | Explaining Residual Variance: Errors in the Model versus Errors in Measurement
PowerPoint version

October 31, 2002
Lecture 16 | The Problem of Sampling Error: An Intuitive Exploration of the Problem
PowerPoint version

November 5, 2002
Lecture 17 | Quantifying Sampling Error: The Standard Error of the Mean
PowerPoint version

S-Plus Quiz

November 7, 2002
Lecture 18 | Quantifying Sampling Error: Confidence Intervals and Forward and Backward Inference
PowerPoint version

November 12, 2002
Lecture 19 | Null Hypothesis Significance Tests: Using Sampling Distributions to Make Decisions about Sampling Error
PowerPoint version

November 14, 2002
Lecture 20| Null Hypothesis Significance Tests: t-tests
PowerPoint version

November 19, 2002
Lecture 21| Null Hypothesis Significance Tests: Analysis of Variance (ANOVA)
PowerPoint version


November 21, 2002
Lecture 22| Null Hypothesis Significance Tests: Chi-Square
PowerPoint version

November 26, 2002
Lecture 23| Decision Errors and the Concept of Statistical Power
PowerPoint version
| joshdata.sav [right-click; select "save target as", change file type to "all files"; save as joshdata.sav; open file in SPSS]

homework due on Dec 3 2002 regarding power analyses for correlations, t-tests, and ANOVA

December 3, 2002
Lecture 24| Some Common Misinterpretations Concerning Significance Tests and P-Values
PowerPoint version

December 5, 2002
Lecture 25| Using Statistical Power and Effect Sizes in Planning Research and in the Evaluation of Existing Research
PowerPoint version

Final Exam | Due Dec 13th