Psychology 543 Research Design and Analysis http://www.uic.edu/classes/psych/psych543/ 2:30 - 4:50 Tuesdays | 3:00 - 4:50 Thursdays 2019 Behavioral Sciences Building

Instructor: R. Chris Fraley
Office: 1050 "A" BSB
fraley@uic.edu | http://www.uic.edu/~fraley

TA: Josh Hemmerich
joshh@uic.edu | http://tigger.uic.edu/~joshh/

Course Description

The content of the course will be organized into three sections. We will begin by discussing ways in which mathematics and statistics can be used to quantify psychological phenomena. We'll review some important ideas regarding psychological measurement, ways of quantifying the properties of common statistical distributions, and ways of quantifying relationships between variables. Next, we'll discuss the role of mathematics and statistics in theory development. One of the points I'll emphasize repeatedly throughout the course is that statistics is not the same thing as "data analysis." Statistics and mathematics should be incorporated into the theoretical stage of the scientific process, not just the data analysis stage. We'll discuss some ways of doing this, focusing primarily on the use of linear equations to model the assumed causal relationships between variables. We'll review methods for constructing these models, and various techniques for estimating their parameters. Finally, we'll discuss ways in which mathematics and statistics can be used to evaluate scientific models. We'll focus on three sources of model misfit: (a) errors in measurement, (b) sampling errors, and (c) fundamental problems in the model per se. For each of these issues, we'll review techniques that are used to handle or minimize these problems (e.g., estimating reliability of scores, designing research in order to minimize the standard error of estimates). Finally, we'll review a set of tools commonly used in psychology, null hypothesis significance tests, designed to handle the problem of sampling error. You will need to be aware of "significance tests" because they are widely used. However, these classic techniques are hotly debated among quantitative psychologists in contemporary psychology, and you will need to be aware of their limitations. We'll discuss both their uses and misuses, and focus on alternative methods for dealing with the problem of sampling error (e.g., parameter estimates, effect sizes, confidence intervals).

Historically, Psych 543 is known as the "ANOVA course" in our department because it is typically taught with a strong emphasis on experimental design. As someone who does not conduct a lot of randomized experiments, I tend to emphasize the modeling of psychological phenomena the way they occur "naturally" (i.e., in their full confounded glory). We will briefly review ANOVA, but you need to know in advance that ANOVA will only constitute a small piece of this course. If you plan to focus on experimental research in your graduate career, I encourage you to take an additional class in ANOVA during your second year.

I have six major objectives for this course:

The first is to provide you with a deep, intuitive understanding of some fundamental concepts in statistics, as they apply to the study of psychological phenomena. We obviously will not have the time (or interest) to review every kind of statistical technique that is used by psychologists. Therefore, I plan to make sure you have a strong handle on the basic mathematical and statistical concepts that are common to many applications.

My second objective is to make sure you can use SPSS to conduct some common statistical analyses. Every two weeks, you will be given a new data set and a set of analytic objectives. You will be required to analyze those data in SPSS, and, as explained below, write both a methods and results section. Midway through the course, I will only provide you with the data; you will need to determine on your own how to analyze the data and summarize them appropriately.

Third, in an attempt to help you "think outside the box", I intend to teach you how to program statistical routines in the S-Plus programming environment. Unless you know a little bit about statistical programming, you will never have the freedom to think creatively about mathematics and statistics, and you'll be constrained to run only the analyses that are available in popular statistical packages, such as SPSS. Every other week you will have to complete a programming exercise for S-Plus (e.g., simulating sampling distributions, bootstrapping confidence intervals). Early in the semester I'll spend time explicitly teaching you how to use S-Plus, and I'll walk you through some of the assignments. Later in the semester I'll simply give you a problem and expect you to solve it on your own by using S-Plus.

Fourth, I want you to become experienced in thinking critically about research design. Unfortunately, many studies that are conducted in psychology are not designed carefully enough to answer the basic questions they set out to answer. By providing you with examples to study and critique--and by encouraging you to think critically about your own research ideas--I hope to provide you with skills that will benefit your research in the long run.

My fifth goal is to make you aware of some basic controversies in the application of statistical tools to psychological science. Students often learn about statistics as if they are learning a set of recipes for data analysis, or worse, a set of facts. Many of the most commonly used statistical tools in psychology have been hotly contested since their inception. In order to be an educated consumer and user of statistical techniques, you need to understand these debates and exactly what certain tools are and are not capable of achieving.

Finally, I plan to help you learn how to write professional methods and results sections. Every two weeks you will be asked to write a methods and results section for a hypothetical set of data. I will grade these fairly rigorously, attending not only to the basic organization of your ideas, but the quality and clarity with which you convey them. Although my primary goal is to help you become great a scientist, my secondary goal is to help you become a great writer.

Textbooks
Hays, W. L. (1994). Statistics (5th ed.). Fort Worth: Harcourt College Publishers.

This text is currently being revised, and the new edition will not be available for another year or two. In the meantime, used copies of the most recent edition are available via many outlets on-line (e.g., www.amazon.com's used book sellers). I encourage you to order a copy and read it at your own pace. I will not be assigning specific chapters from this text, but you will need a fairly comprehensive statistics text on your reference shelf if you plan to pursue a PhD in psychological science. [link to Hays text on amazon.com]

Salsburg, D. (2001). The lady tasting tea: How statistics revolutionized science in the twentieth century. NY: W. H. Freeman and co.

This book provides a wonderful overview of the history of statistics, with special attention to the major players and the key controversies. Why am I assigning this book? Statistics courses are often taught in a passion-free, recipe-like manner. This way of characterizing the development of modern statistics overlooks the fact that statisticians have been--and are--quite divided over some of the concepts that psychologists accept as "facts." This book will help make you aware of these debates, and, I hope, facilitate your ability to think critically about "statistical conventions." [link to Salsburg book on amazon.com]

Krause, A., & Olson, M. (2002). The basics of S-Plus. NY: Springer-Verlag.

This book is recommended (not required) if you would like a useful text on programming in S-Plus. [link to Krause-Olson book on amazon.com]

Grading
Grades will be based on your class participation, the quality of your assignments (S-Plus exercises, SPSS exercises, and Methods/Results exercises), quizzes, and exams.

Students

Updates
Assignments, datasets, notes, and updates will be listed here throughout the semester. Please treat this webpage as your primary syllabus.

Lecture Overheads
Overheads and (some) notes from class discussions will be available here. You may click on the lecture title to view an HTML version of the overheads. Alternatively, you may right-click on the PowerPoint option and select "save target" to save the original PowerPoint files.