February 9, 2004
A: FROM MENTOR JOAN LUSK IN RI
This is really not my field, but I think it would be worthwhile for you to
calculate how large a sample size you'd need to make the 'insignificant'
results you found become statistically significant at the desired level, if
in fact the result was real but not statistically significant. That way you
could decide whether a larger study would be practical.

I wonder if you've thought about the interplay between a background color
and the colors within the picture. Would a clashing color make a more
memorable presentation than a harmonious background, independent of what
the background color was? Do people remember what the background color
was? Some people have strong likes and dislikes about color - does that
affect memory? I have a friend who can't stand blue and talks about
getting rid of it wherever she sees it - won't stay in a blue hotel room -
I bet she'd remember blue (but forget whatever the picture was!) Now that
I think of it, my friend hated living in California because it never rained
- I bet the sky was too blue. But I digress.... and there are also
cultural differences in the emotional meaning of colors - white can mean
purity or death, red can mean joy or blood... these emotional responses
could affect memory, I'd bet.
********************
A: FROM MENTOR DENISE HARBERT IN IL

Wow! How very cool! I could probably write a book on this question because
my background is perfectly matched to this experiment! I studied statistics,
perception psychology, and graphics. I apologize for how long this is going
to be, but I think I can help you! You have a few different problems that you
may or may not be able to address.

First, I think you have a power problem. I believe that color does impact a
person's ability to process information and remember it. However, the
magnitude of the effect color has on memory might be so small that it is
difficult to detect statistically. In scientific terms, you may not have
enough statistical power to detect the effect color has on memory.
Statistical power is affected by the:
1. magnitude of the effect size (actual difference in memory due to color)
2. variability in your sampling distribution (how much your data vary)
3. size of your rejection region (alpha)
4. type of alternative hypothesis (one-sided or two-sided)
5. statistical test you use to analyze your data (parametric or
non-parametric)

Here is how you might be able to address each of these things:

1. Rather than comparing different colors to each other and looking for
differences between colors, try a "control group" of no color (white). For
example, the effect size between dark purple and white may be larger than the
effect size between dark purple and yellow. Larger effect sizes are easier to
detect.

2. There are generally 3 ways to reduce variability:
2a) standardize the data collection process. Don't alternate between you
and your partner to give the tests. Only one of you or both of you together
should always give the tests in the same way to avoid confounding your
personality differences with your results. Always read the exact same set of
directions to every participant. Try to give your tests under the same
situations, like same time of day, same type of room (whether windows or not),
same amount of time available (a person who is late for an appointment will be
less focused on your task), etc. You may try a "double blind" design. You
know what you are trying to determine, so you may be subconsciously conveying
that to your participants. Try getting a third person involved who has no
idea what you are doing and ask that person to administer the tests. (You
can offer the person 3rd billing as an author in your project's
presentation.)
2b) increase your sample size: Test more than 20 people.
2c) dependent design: You may have already done this. Try to figure out a
way to have each person tested with each color combination, so you can
eliminate the variability between people and only focus on the variability
between colors. You may want to schedule test sessions with each person
spaced a week apart so there is less of a "learning" factor. You can reduce
the number of colors to make this more reasonable.

3. Increase the size of your rejection region. Most people choose alpha = .05
because that is the standard chosen in most statistics textbooks. However, a
chemist working with molecules or a physicist working with forces may choose
an alpha = .01 or .001 because molecules and forces should behave with less
variability than other types of experiments, so differences should be more
easily detectable. Likewise, experiments involving people should have much
more variability and should be more difficult to detect differences.
Psychologists often choose alpha = .10 or may go even larger depending on the
topic. (Warning: large alphas should never be used if your null hypothesis is
dangerous or life threatening. You want to be very sure you are proving that
something is safe, so you do not want to conclude something is safe when it is
actually unsafe.)

4. If it is reasonable to use a one-sided alternative hypothesis, then doing
so will give you more power than using a two-sided alternative. For example,
if you want to prove that color affects memory, then that is a two-sided
alternative because color may improve memory or hinder it. Either direction
would be of interest to you. If you are only interested in showing whether or
not color hinders memory, then that would be a one-sided alternative. (Note
that not all statistical tests allow you to select the direction of your
alternative. Some tests are automatically two-sided by the nature of how they
were mathematically derived.)

5. This deals mainly with what assumptions you can satisfy. Most parametric
tests require that your data or your errors (residuals) be IID normal
(independent, identically distributed, normal distribution). This
distribution is very specific, requiring a bell shape that follows a specific
equation with a fixed mean and constant variance. It is often very difficult
to satisfy this requirement. If your data do not satisfy this requirement,
then a parametric test (like the independent sample t-test) is probably not
appropriate and a non-parametric test (like the rank-sum test) would probably
be more valid. On the other hand, if this assumption is satisfied, then the
parametric test has more power than the non-parametric test.

An excellent book for learning about statistics in a psychology context is
called "Learning from Data: An Introduction to Statistical Reasoning" by
Arthur M. Glenberg. He was one of my college professors and wrote the book
for Freshman and Sophomore college students to give them a "fighting chance"
of understanding statistics. He also teaches courses in learning and memory,
so the book is written in such a way that concepts are reinforced and built
upon throughout the book. Concepts that most statistics books skim over or
assume the reader already knows are thoroughly explained in different ways.
This makes a difficult subject seem easy and I cannot recommend any statistics
book more highly!

Aside from power, another thing you might want to focus on is altering your
experimental design. It seems like you have quite a few factors that could be
confounded with what you are trying to measure, namely the effect of color on
memory. You are asking people to write pictures in an order that you have
switched each time. Although switching order does reduce the "learning
effect" (people learn as the test goes on so they perform better on the last
test than on the first one), it also introduces a new effect: the interaction
of order with the color. Maybe some colors make it easier or harder to
correctly remember different orders. Also, what are your 20 pictures and
could they affect the participant? For example, if you give a chess master 20
pictures of chess pieces to order with a purple background, then she may do
that more quickly than when you show her 20 pictures of dogs with a yellow
background. Rather than measuring the order of various pictures, could you
measure something else?

Have you done a literature review on color and perception? I imagine that
there are several perception psychology research papers on color that might
help you narrow your question. One statistics author you may find helpful is
Edward Tufte, who wrote a book called "The Visual Display of Quantitative
Information". This book is clear, short, and easy enough for any high school
student to read. Tufte wrote extensively on graphics and how various elements
within a graph can make the data easier or harder to understand. Tufte showed
examples of how color can make it virtually impossible to see data. In a more
recent book, he provided examples of when color is an effective graphical
tool. Tufte believes that color works well in a graphic when it quickly
identifies problem areas by contrasts. For example, brain scans can show up
with red and dark orange near a tumor, surrounded by dark blue along the edges
of the skull. Color works poorly in a graphic when a person has to look
between the graphic and the legend repeatedly in order to decipher what
they are looking at. Maybe you could design an experiment around that. For
example, find a graphing software that allows you to put different colors on a
graphic. (Microsoft MapPoint allows you to select one of about a dozen
different color series to place on a map of the U.S. One of the series has
several different colors.) Make the same graph with several different colors
and design an experiment around how long it takes participants to answer quiz
questions about the graphic. Or, give them more questions than they could
possibly answer and see how many they can answer within a fixed time period.

Colorblindness is also a problem that you should be sure to account for among
your participants. Colorblindness is one of the only gender differences that
has been proven to be genetically caused. The gene related to color is on the
X chromosome on the leg that is "missing" from the Y chromosome. Women have
two X chromosomes and men only have one. Thus, if a woman has the color
defect on one of her X's, she will still be able to see colors perfectly as
long as her other X does not have the defect. Men only have one X, so if a
man has the defect on his X, he will be at least partially colorblind. Thus,
men are far more likely to be colorblind than women. When I researched this
during my thesis on graphics, I found papers that indicated that up to 10% of
all men have at least some degree of colorblindness. If half of your 20
participants were male, then probability tells me that one of them probably
had some colorblindness, which could have notably skewed your results. You
may want to give your participants a colorblindness test before you give them
your test. Or, you may want to limit your experiment to females only.

Well, that is probably more than you ever wanted to know about color! But
good luck and I hope you find something statistically significant on your next
try!!!