Use of the Pearson correlation co-efficient is common in genomics and bioinformatics, which is OK as it goes (I have used it extensively myself), but it has some major drawbacks – the major one being that Pearson can produce large coefficients in the presence of very large measurements.

This is best shown via example in R:

```
# let's correlate some random data
g1 <- rnorm(50)
g2 <- rnorm(50)
cor(g1, g2)
# [1] -0.1486646
```

So we get a small, -ve correlation from correlating two sets of 50 random values. If we ran this 1000 times we would get a distribution around zero, as expected.

Let's add in a single, large value:

```
# let's correlate some random data with the addition of a single, large value
g1 <- c(g1, 10)
g2 <- c(g2, 11)
cor(g1, g2)
# [1] 0.6040776
```

Holy smokes, all of a sudden my random datasets are positively correlated with r>=0.6!

It's also significant.

```
> cor.test(g1,g2, method="pearson")
Pearsons product-moment correlation
data: g1 and g2
t = 5.3061, df = 49, p-value = 2.687e-06
alternative hypothesis: true correlation is not equal to 0
95 percent confidence interval:
0.3941015 0.7541199
sample estimates:
cor
0.6040776
```

So if you have used Pearson in large datasets, you will almost certainly have some of these spurious correlations in your data.

How can you solve this? By using Spearman, of course:

```
> cor(g1, g2, method="spearman")
[1] -0.0961086
> cor.test(g1, g2, method="spearman")
Spearmans rank correlation rho
data: g1 and g2
S = 24224, p-value = 0.5012
alternative hypothesis: true rho is not equal to 0
sample estimates:
rho
-0.0961086
```

27th April 2016 at 10:46 am

I would suggest Kendall rank correlation, rather than Spearman’s rho. Kendall rank correlation is a non-parametric test that measures the strength of dependence between two variables.

The main advantages of using Kendall’s tau are as follows:

1. The distribution of Kendall’s tau has better statistical properties.

2. The interpretation of Kendall’s tau in terms of the probabilities of observing the agreeable (concordant) and non-agreeable (discordant) pairs is very direct. It tells you the difference between the probability that the two variables are in the same order in the observed data versus the probability that the two variables are in different orders.

3. Dave Howell in Statistical Methods for Psychology, concludes that Kendall’s tau is generally preferred over Spearman’s rho, “because it is a better estimate of the corresponding population parameter, and its standard error is known.”

4. Roger Newson has argued for the superiority of Kendall’s τ (tau) over Spearman’s rho as a rank-based measure of correlation in a paper: Newson R. Parameters behind “nonparametric” statistics: Kendall’s tau,Somers’ D and median differences. Stata Journal 2002; 2(1):45-64. (http://www.stata-journal.com/article.html?article=st0007). He references (on p47) to Kendall & Gibbons (1990) as arguing that “…confidence intervals for Spearman’s rho are less reliable and less interpretable than confidence intervals for Kendall’s τ-parameters, but the sample Spearman’s rho is much more easily calculated without a computer” (which is no longer of much importance of course).

Calculation of Kendall’s tau in R: cor.test(x, y, method=”kendall”)

27th April 2016 at 1:32 pm

Fair enough.

Two comments:

1- not everyone works with 50 genes:

a <- rnorm(30000)

b <- rnorm(30000)

cor(a, b, method = "pearson")

[1] 0.0009655594

cor(c(a, 10), c(b, 11))

[1] 0.004605644

Relatively massive impact, but still low PCC value.

2- for some applications, time maters:

system.time(cor(a, b, method = "pearson"))

user system elapsed

0 0 0

system.time(cor(a, b, method = "spearman"))

user system elapsed

0.08 0.01 0.09

system.time(cor(a, b, method = "kendall"))

user system elapsed

111.48 0.09 116.69

27th April 2016 at 1:49 pm

Looking at the data is also a good way to prevent this king of things.

4th May 2016 at 1:37 pm

Very interesting post and comments.

@Guillaume

Good highlight on limitations.

Not everybody is working on 30.000 samples 😉

5th May 2016 at 1:11 am

It isn’t 30,000 samples that is the problem—it is 30,000 choose 2 hypotheses!

5th May 2016 at 9:54 pm

Sorry, I don’t get your point concerning 30,000.

It sounds clear that there no valuable impact on Pearson’s correlation of 2 nicely correlated values among 30,000. Among 50 values, the impact is clear.

Let’s say we compare 2 genes measured on 50 samples. On one side, I may consider that I am not interested in the occurrence of 2 high values on the same sample whereas values of the other samples are low. Spearman is then OK. On the other side, if this occurrence might be of interest, Pearson sounds OK.

IMHO, it’s a question of choice, between robustness and sensitivity, depending on the question addressed to the data.

http://rpubs.com/SamGG/PearsonVsSpearmanCorrelation

6th May 2016 at 4:37 am

The big question is whether the two genes were chosen before looking at the data, in which case correlation on 50 samples can be highly informative, or after looking at the data (picking the highest correlated pair of genes), in which case 50 samples is nowhere near enough to say anything meaningful.

6th May 2016 at 11:02 am

I get your point now. Confirmatory experiment is necessary indeed.

If the initial point raised by this post concerns the use of Pearson’s correlation in hierarchical clustering, I think there is no risk of getting wrong results. Concerning sample correlation, one over thousands of genes does not change correlation a lot, as shown. Concerning gene correlation, groups of samples consist in ten samples at least, and the correlation between genes is safe also.

If the point concerns picking best correlations among a matrix of gene correlations in order to induce some networks for example, this post is a valuable warning.

In all the cases, it’s a good discussion.