Psychology 543 Research Design and Analysis http://www.uic.edu/classes/psych/psych543/ 2:30 - 4:50 Tuesdays | 3:00 - 4:50 Thursdays 2019 Behavioral Sciences Building

Instructor: R. Chris Fraley
Office: 1050 "A" BSB
fraley@uic.edu | http://www.uic.edu/~fraley

TA: Josh Hemmerich
joshh@uic.edu
| http://tigger.uic.edu/~joshh/

Course Description

The content of the course will be organized into three sections. We will begin by discussing ways in which mathematics and statistics can be used to quantify psychological phenomena. We'll review some important ideas regarding psychological measurement, ways of quantifying the properties of common statistical distributions, and ways of quantifying relationships between variables. Next, we'll discuss the role of mathematics and statistics in theory development. One of the points I'll emphasize repeatedly throughout the course is that statistics is not the same thing as "data analysis." Statistics and mathematics should be incorporated into the theoretical stage of the scientific process, not just the data analysis stage. We'll discuss some ways of doing this, focusing primarily on the use of linear equations to model the assumed causal relationships between variables. We'll review methods for constructing these models, and various techniques for estimating their parameters. Finally, we'll discuss ways in which mathematics and statistics can be used to evaluate scientific models. We'll focus on three sources of model misfit: (a) errors in measurement, (b) sampling errors, and (c) fundamental problems in the model per se. For each of these issues, we'll review techniques that are used to handle or minimize these problems (e.g., estimating reliability of scores, designing research in order to minimize the standard error of estimates). Finally, we'll review a set of tools commonly used in psychology, null hypothesis significance tests, designed to handle the problem of sampling error. You will need to be aware of "significance tests" because they are widely used. However, these classic techniques are hotly debated among quantitative psychologists in contemporary psychology, and you will need to be aware of their limitations. We'll discuss both their uses and misuses, and focus on alternative methods for dealing with the problem of sampling error (e.g., parameter estimates, effect sizes, confidence intervals).

Historically, Psych 543 is known as the "ANOVA course" in our department because it is typically taught with a strong emphasis on experimental design. As someone who does not conduct a lot of randomized experiments, I tend to emphasize the modeling of psychological phenomena the way they occur "naturally" (i.e., in their full confounded glory). We will briefly review ANOVA, but you need to know in advance that ANOVA will only constitute a small piece of this course. If you plan to focus on experimental research in your graduate career, I encourage you to take an additional class in ANOVA during your second year.

I have six major objectives for this course:

The first is to provide you with a deep, intuitive understanding of some fundamental concepts in statistics, as they apply to the study of psychological phenomena. We obviously will not have the time (or interest) to review every kind of statistical technique that is used by psychologists. Therefore, I plan to make sure you have a strong handle on the basic mathematical and statistical concepts that are common to many applications.

My second objective is to make sure you can use SPSS to conduct some common statistical analyses. Every two weeks, you will be given a new data set and a set of analytic objectives. You will be required to analyze those data in SPSS, and, as explained below, write both a methods and results section. Midway through the course, I will only provide you with the data; you will need to determine on your own how to analyze the data and summarize them appropriately.

Third, in an attempt to help you "think outside the box", I intend to teach you how to program statistical routines in the S-Plus programming environment. Unless you know a little bit about statistical programming, you will never have the freedom to think creatively about mathematics and statistics, and you'll be constrained to run only the analyses that are available in popular statistical packages, such as SPSS. Every other week you will have to complete a programming exercise for S-Plus (e.g., simulating sampling distributions, bootstrapping confidence intervals). Early in the semester I'll spend time explicitly teaching you how to use S-Plus, and I'll walk you through some of the assignments. Later in the semester I'll simply give you a problem and expect you to solve it on your own by using S-Plus.

Fourth, I want you to become experienced in thinking critically about research design. Unfortunately, many studies that are conducted in psychology are not designed carefully enough to answer the basic questions they set out to answer. By providing you with examples to study and critique--and by encouraging you to think critically about your own research ideas--I hope to provide you with skills that will benefit your research in the long run.

My fifth goal is to make you aware of some basic controversies in the application of statistical tools to psychological science. Students often learn about statistics as if they are learning a set of recipes for data analysis, or worse, a set of facts. Many of the most commonly used statistical tools in psychology have been hotly contested since their inception. In order to be an educated consumer and user of statistical techniques, you need to understand these debates and exactly what certain tools are and are not capable of achieving.

Finally, I plan to help you learn how to write professional methods and results sections. Every two weeks you will be asked to write a methods and results section for a hypothetical set of data. I will grade these fairly rigorously, attending not only to the basic organization of your ideas, but the quality and clarity with which you convey them. Although my primary goal is to help you become great a scientist, my secondary goal is to help you become a great writer.

Textbooks
Hays, W. L. (1994). Statistics (5th ed.). Fort Worth: Harcourt College Publishers.

This text is currently being revised, and the new edition will not be available for another year or two. In the meantime, used copies of the most recent edition are available via many outlets on-line (e.g., www.amazon.com's used book sellers). I encourage you to order a copy and read it at your own pace. I will not be assigning specific chapters from this text, but you will need a fairly comprehensive statistics text on your reference shelf if you plan to pursue a PhD in psychological science. [link to Hays text on amazon.com]

Salsburg, D. (2001). The lady tasting tea: How statistics revolutionized science in the twentieth century. NY: W. H. Freeman and co.

This book provides a wonderful overview of the history of statistics, with special attention to the major players and the key controversies. Why am I assigning this book? Statistics courses are often taught in a passion-free,
recipe-like manner. This way of characterizing the development of modern statistics overlooks the fact that statisticians have been--and are--quite divided over some of the concepts that psychologists accept as "facts." This book will help make you aware of these debates, and, I hope, facilitate your ability to think critically about "statistical conventions." [link to Salsburg book on amazon.com]

Krause, A., & Olson, M. (2002). The basics of S-Plus. NY: Springer-Verlag.

This book is recommended (not required) if you would like a useful text on programming in S-Plus. [link to Krause-Olson book on amazon.com]

Grading
Grades will be based on your class participation, the quality of your assignments (S-Plus exercises, SPSS exercises, and Methods/Results exercises), quizzes, and exams.

Students

 Name e-mail Adabel Lee adabel@uic.edu Elgiz Bal ebal@psych.uic.edu Zia Lakdawalla zlakda1@uic.edu Aarati Kasturirangan aarati@uic.edu Sandhya Krishnan srk@uic.edu Ben Jee ben.jee@utoronto.ca Katherine Bailey kbaile3@uic.edu Aliza Silver aliza@uic.edu Dana Rusch drusch1@uic.edu Daniel Evatt evatt@uic.edu

Updates
Assignments, datasets, notes, and updates will be listed here throughout the semester. Please treat this webpage as your primary syllabus.

 August 29, 2002 Next Tuesday, Sept 3, we will begin class at 3:00 instead of 2:30. August 29, 2002 Here are some links to some useful UIC resources: PsychINFO Electronic journals UIC library | catalog (for book and journal call numbers and locations) UIC web mail UIC psychology department September 12, 2002 Next Tuesday we will discuss the article that was distributed in class this past Tuesday. Be prepared to provide a critical analysis of the research questions, design, and analyses. Also, please turn in a one-page summary of your critique. Please proofread your writing; I will be attending to the quality of your writing. September 12, 2002 As the semester progresses, you'll need to rely on the computers in 2019 in order to complete assignments, practice, etc. I plan to reserve the classroom for these purposes for the following times: Mondays : after 5 Tuesdays : after our class Wednesdays : after 4 Thursdays : after our class If you are using the room during one of these times, and if you're the last one to leave, please be sure the lights are out and that the door is closed. October 2, 2002 Next Tuesday (Oct 8) we will discuss the article that was distributed in class this past Tuesday. Be prepared to provide a critical analysis of the research questions, design, and analyses. Also, please turn in a one-page summary of your critique. Please proofread your writing; I will be attending to the quality of your writing. October 9, 2002 Next Tuesday (Oct 15) we will have a quiz in class. Please make sure you're comfortable with all the material that we've dicussed in class up to this point.

Lecture Overheads
Overheads and (some) notes from class discussions will be available here. You may click on the lecture title to view an HTML version of the overheads. Alternatively, you may right-click on the PowerPoint option and select "save target" to save the original PowerPoint files.

 Lecture 1 | August 29, 2002 The Use of Mathematics and Statistics to Understand the Behavior of Natural Systems PowerPoint version S-Plus scripts: orbits2.SSC [right-click to save to disk] Lecture 2 | September 3, 2002 Quantification and Scales of Measurement PowerPoint version Introduction to S-Plus Lecture 3 | September 5, 2002 Visually Summarizing Score Distributions PowerPoint version Creating a web page | Using FTP [powerpoint file]| Making a web page from scratch [PDF file] Lecture 4 | September 9, 2002 Quantitative Summaries of Score Distributions: Central Tendency PowerPoint version Dreamweaver practice & creating web pages | Download NsKit (with WS FTP) here Lecture 5| September 12, 2002 Quantitative Summaries of Score Distributions: Spread and Dispersion PowerPoint version Lecture 6| September 17, 2002 Comparing Scores for Different Variables: The Use of Standardized Scores and Standardized Metrics PowerPoint version September 19, 2002 Deriving Quantitative Indices of Association: Spearman's Rank-Order Correlation September 24, 2002 Deriving Quantitative Indices of Association: Spearman's Rank-Order Correlation [continued discussion] Lecture 7 | Quantifying the Association Between Two Variables: Covariance PowerPoint version September 26, 2002 Deriving Quantitative Indices of Association: Correlation Lecture 8| Quantifying the Association Between Two Variables: Correlation PowerPoint version dataset1.sav [right-click; select "save target as", change file type to "all files"; save as dataset1.sav; open file in SPSS] October 1, 2002 Lecture 9 | Mathematical Modeling of Theoretical Relationships: Function Forms and their Specification PowerPoint version assignment due: derivation of the correlation using the squared difference of z-scores rather than the products of z-scores October 3, 2002 Lecture 10 | Least Squares: Estimating the Parameters of Simple Linear Models PowerPoint version October 8, 2002 Lecture 11 | Evaulating Model Fit: R-Squared PowerPoint version October 10, 2002 Lecture 12 | Using R-Squared to Compare the Fit of Alternative Models PowerPoint version splustutorial1.SSC [right-click; select "save target as", change file type to "all files"; save as splustutorial1.ssc; open file in S-Plus] October 15, 2002 Lecture 13 | Confounds and Complex Causal Structures: Tracing Rules and Methodological and Statistical Control PowerPoint version dataset2.sav | dataset3.sav [right-click; select "save target as", change file type to "all files"; save as dataset2.sav and dataset3.sav, respectively; open file in SPSS] October 22, 2002 Lecture 14 | Additive vs. Nonadditive Effects: Interactions with Binary and Continuous Predictors PowerPoint version October 29, 2002 Lecture 15 | Explaining Residual Variance: Errors in the Model versus Errors in Measurement PowerPoint version October 31, 2002 Lecture 16 | The Problem of Sampling Error: An Intuitive Exploration of the Problem PowerPoint version November 5, 2002 Lecture 17 | Quantifying Sampling Error: The Standard Error of the Mean PowerPoint version S-Plus Quiz November 7, 2002 Lecture 18 | Quantifying Sampling Error: Confidence Intervals and Forward and Backward Inference PowerPoint version November 12, 2002 Lecture 19 | Null Hypothesis Significance Tests: Using Sampling Distributions to Make Decisions about Sampling Error PowerPoint version November 14, 2002 Lecture 20| Null Hypothesis Significance Tests: t-tests PowerPoint version November 19, 2002 Lecture 21| Null Hypothesis Significance Tests: Analysis of Variance (ANOVA) PowerPoint version Quiz November 21, 2002 Lecture 22| Null Hypothesis Significance Tests: Chi-Square PowerPoint version November 26, 2002 Lecture 23| Decision Errors and the Concept of Statistical Power PowerPoint version | joshdata.sav [right-click; select "save target as", change file type to "all files"; save as joshdata.sav; open file in SPSS] homework due on Dec 3 2002 regarding power analyses for correlations, t-tests, and ANOVA December 3, 2002 Lecture 24| Some Common Misinterpretations Concerning Significance Tests and P-Values PowerPoint version December 5, 2002 Lecture 25| Using Statistical Power and Effect Sizes in Planning Research and in the Evaluation of Existing Research PowerPoint version Final Exam | Due Dec 13th