The Chi-Square Independence test is used a complex tool (Excel recommended!) to determine if two attributes of the same object are correlated or independent from each other, with a high level of certainty (>95%). The Chi-Square coefficient is given by the formula above using (obs)erved frequencies and (exp)ected frequencies of the values in the table.

Verify the following checklist before using the Chi-Square Independence Test:

1. The sample must include at least 20 inviduals (the larger the sample the better)
   
2. The data must be exact values, NOT percentages or frequencies
   
3. Both observations MUST be independent from each other: one should not be the direct cause of the other (e.g. do not use this test to compare HDI and Life Expectancy, since the HDI is a composite index which is partly based on Life Expectancy!)
   
4. You are only trying to compare TWO attributes of the same object/person at a time (e.g. income AND gender, or temperature AND elevation)
   
5. Each attribute (gender, nationality, preference, etc) is divided into at least TWO possibilities, preferably more whenever possible (e.g. nationality: US, French, British, Japanese, etc). A table with only 2 lines and 2 columns might not lend itself to a significant Chi-Square independence test

The Chi-Square independence test determines whether two attributes of one object are related or independent from each other:

1. Verify that you sample meets the 5 criteria above and formulate the null hypothesis
   
2. Arrange the sample data in a contingency table (=table of frequencies) (see tutorial below)
   
3. Calculate the Chi-Square correlation coefficient
   
4. Use a table to find the Chi-Square critical value (depending on your sample size and required confidence level)
5. If your Chi-Square value ("Chi-Square statistic") > critical Chi-Square value => there is a correlation, based on the sample considered

The sample used is fundamental because if we had used a different sample (ex: smaller, larger, collected another day, etc), we might prove that the correlation is actually different (weaker or stronger). Typically, the bigger the sample, the more accurate the Chi-Square independence test will be.

Watch these two tutorials to learn how to run and interpret the Chi-Square Independence Test without Excel, or check out the examples below

---

Example 1: Is the nationality of foreign tourists visiting SF related to which tourist attraction they liked most?

• Objects: tourists in SF
• Attribute #1: nationality (French, German or Italian)
• Attribute #2: preferred SF attraction (De Young, Mission Dolores, Fisherman's Wharf, Other)

(note: numbers used for this example are ficticious)

Step 1: Test criteria and null hypothesis

• Comparing the nationality and individual preference in someone meets the criteria (see above) to run the Chi-Square test
• The Null Hypothesis H(0) is "the nationality of foreign tourists visiting SF is INDEPENDENT from which tourist attraction they like most in SF"

Step 2: Table of "observed frequencies"
Create a table of the numbers you have collected (e.g. interviews of tourists in the street) in a table combining attribute #1 (columns) and attribute #2 (rows) as follows:

OBSERVED FREQUENCIES	French	German	Italian	Total
De Young Museum	25	14	5	44
Mission Dolores	12	27	12	51
Fisherman's Wharf	3	13	19	35
Other	14	8	15	37
Total	54	62	51	Σ = 167

Step 3: Table of "expected frequencies"
Using the totals of each line and column, calculate what frequencies would be expected if the distribution was completely random (note: if the expected frequencies are smaller than 5, it usually means that the sample is too small for the Chi-Square independence test to be meaningful). To do that, just multiply the total of a line by the total of a column, and divide by the total of the whole table. For example, 44 of all 167 tourists liked the De Young most while 54 of all 167 tourists are French. Therefore, if we did not have the details about who preferred what, we could expect the average number of tourists who are French AND who preferred the De Young Museum to be: 44 x 54 / 167 = 14.2 tourists

EXPECTED FREQUENCIES	French	German	Italian	Total
De Young Museum	44x54/167 = 14.2	44x62/167 = 16.3	44x51/167 = 13.4	44
Mission Dolores	51x54/167 = 16.5	51x62/167 = 18.9	51x51/167 = 15.6	51
Fisherman's Wharf	35x54/167 = 11.3	35x62/167 = 13.0	35x62/167 = 10.7	35
Other	37x54/167 = 12.0	37x62/167 = 13.7	37x51/167 = 11.3	37
Total	54	62	51	Σ = 167

Step 4: Run the Chi-Square test with Excel:
If you are not using Excel, go to step 5 to learn how to use the Chi-Square formula (see above). With Excel, simply use the formula =CHITEST and select the array of observed frequencies (blue) and the array of expected frequencies (yellow) (do NOT select the total row and column with the totals). Using the tables above, the result is: CHITEST = 0.000003 = 0.0003%
This means that there is only a 0.0003% chance that the Null Hypothesis is true and that the two attributes (nationality and preferred tourist attraction) are INDEPENDENT from each other. In other words, there is a 99.9997% chance that the two attributes are related. Anything above 95% being almost certain, we have to conclude that the two attributes are almost certainly related: based on this sample, certain nationalities tend to prefer specific SF tourist attractions. Further analysis may try to explain why French tourists tend to prefer the De Young while Germans tend to prefer the Mission Dolores: cultural reasons? Social reasons? Tour operators?

Step 5: Calculate the Chi-Square value "manually":
Create a third table: use observed (O) and expected (E) values found above to calculate the value in each cell of the new table, using the Chi-Test formula (O-E)2/E, as follows:

CHI-SQUARE	French	German	Italian	Total
De Young Museum	(O-E)²/E = 8.15	(O-E)²/E = 0.33	(O-E)²/E = 5.30	13.78
Mission Dolores	(O-E)²/E = 1.22	(O-E)²/E = 3.44	(O-E)²/E = 0.82	5.48
Fisherman's Wharf	(O-E)²/E = 6.11	(O-E)²/E = 0.00	(O-E)²/E = 6.46	12.57
Other	(O-E)²/E = 0.35	(O-E)²/E = 2.40	(O-E)²/E = 1.21	3.96
Total	15.83	6.17	13.79	Σ = 35.79

Step 6: degrees of freedom, confidence level and critical Chi-Square values:
The number of “degrees of freedom” in the sample combines the number of nationalities (3 choices/columns) and attractions (4 choices/rows) as follows:
df = (# of nationalities – 1) x (# of attractions – 1) = (3-1)x(4-1) = 6
This sample therefore has 6 "degrees of freedom".

The "confidence level” is the minimum probability (p) over which you can safely say that the two attributes examined are related (usually 95%, 99% or 99.9%). Vice versa, the “level of significance” is the maximum level of error (e) you want to allow for the test: e = 100% – p = 1 – p

• If p = 95% (= 0.95), it means that the maximum level of error (e) you want to allow is e = 100% - 95% = 5% (= 0.05)
• If p = 99% (= 0.99), it means that the maximum level of error (e) you want to allow is e = 100% - 99% = 1% (= 0.01)
• If p = 99.9% (= 0.999), it means that the maximum level of error (e) you want to allow is e = 100% - 99.9% = 0.1% (= 0.001)

The critical values of Chi-Square are the benchmarks against which to compare the Chi-Square value you have found above. The critical values are set, and depend only upon the degrees of freedom of the sample and the minimum confidence level you have decided to require. Choose ONE of the following methods to calculate the critical values of Chi-Square:

• Excel: the formula =CHIINV(e,df), where (e) is level of error allowed and (df) is the number of degrees of freedom gives you the following critical values: 22.5 (for e = 0.001, or 99.9% confidence), 16.8 (for e = 0.01, or 99% confidence), and 12.6 (for e = 0.05, or 95% confidence)
  
• Online Chi-Square critical value calculator: enter the degrees of freedom in the top field and the level of error allowed (e) in bottom right, then click “compute” to show the critical value in the bottom left field. If the Chi-Square coefficient of your sample is in the shaded blue area, then the attributes are related beyond the required degree of certainty
• Online table of Chi-Square critical values

Step 7: Interpret your results
ou must compare the Chi-Square coefficient you calculated (step 5) with the critical value of Chi-Square (step 6):

• If Chi-Square > critical value, the null hypothesis can be rejected and that the two attributes are related beyond the confidence level that you have set
  
• If Chi-Square < critical value, the confidence level is not met and the test is inconclusive: the two attributes are probably NOT related, but you need more information to confirm (larger sample for example)
  
• If Chi-Square = 0, the null hypothesis is true and the two attributes are completely NOT related

If you chose a 99.9% confidence level, the Chi-Square critical value (for 6 degrees of freedom) is 22.5. The Chi-Square value of the sample is 35.79: we can therefore reject the null hypothesis and conclude, with a 99.9% certainty, based on this sample, that "the nationality of tourists who visit SF and the attraction they prefer in SF is correlated". Note however that the Chi-Square test only measures the correlation: further analysis would be needed to explain its reasons. Warning: if your sample is small, it is unreliable to generalize your conclusion to all tourists, since a larger sample might yield different observed/expected frequencies...

Example 2: Is the nationality of foreign tourist related to which neighborhood they decide to stay in while in SF?

• Objects: tourists in SF
• Attribute #1: nationality (French, German or Italian)
• Attribute #2: neighborhood chosen to stay in SF (Tenderloin, Marina, Other)

(note: numbers used for this example are ficticious)

Step 1: Test criteria and null hypothesis

• Comparing the nationality and individual preference in someone meets the criteria (see above) to run the Chi-Square test
• The Null Hypothesis H(0) is "the nationality of foreign tourists visiting SF is INDEPENDENT from where tourists decide to stay in SF"

Step 2: Table of "observed frequencies"
Create a table of the numbers you have collected (e.g. interviews of tourists in the street) in a table combining attribute #1 (columns) and attribute #2 (rows) as follows:

OBSERVED FREQUENCIES	French	German	Italian	Total
Tenderloin	48	50	40	138
Marina	3	10	9	22
Other	3	2	2	7
Total	54	62	51	Σ = 167

Step 3: Table of "expected frequencies"
(see step 3 of example 1)

EXPECTED FREQUENCIES	French	German	Italian	Total
Tenderloin	138x54/167 = 44.6	138x62/167 = 51.2	138x51/167 = 42.1	138
Marina	22x54/167 = 7.1	22x62/167 = 8.2	22x51/167 = 6.7	22
Other	7x54/167 = 2.3	7x62/167 = 2.6	7x51/167 = 2.6	7
Total	54	62	51	Σ = 167

Step 4: Run the Chi-Square test with Excel:
If you are not using Excel, go to step 5 to calculate the Chi-Square value using the formula above. With Excel, simply use the formula =CHITEST and select the table of observed frequencies and the table of expected frequencies (do NOT select the total row and column with the totals). Using the tables above, the result is: CHITEST = 0.36 = 36%
This means that there is a 36% chance that the Null Hypothesis is true and that the two attributes (nationality and neighborhood of hotel) are INDEPENDENT from each other. In other words, there is only a 64% chance that the two attributes are related, well below the 95% confidence level required to make any statistical calculation reliable: we have to conclude that the Null Hypothesis can NOT be rejected, and that the two attributes are almost certainly independent from each other. Based on this sample, the nationality of tourists does not seem to be correlated with where they choose to stay while in San Francisco. Since most tourists appear to choose to stay in the Tenderloin over the Mission regardless of their nationality, further analysis may try to explain this phenomenon: Cost? Accessibility? Hotel chains? Number of hotels?

Step 5: Calculate the Chi-Square value "manually":
Create a third table: use observed (O) and expected (E) values found above to calculate the value in each cell of the new table, using the Chi-Test formula (O-E)2/E, as follows:

CHI-SQUARE	French	German	Italian	Total
Tenderloin	(O-E)²/E = 0.26	(O-E)²/E = 0.03	(O-E)²/E = 0.11	0.40
Marina	(O-E)²/E = 2.38	(O-E)²/E = 0.41	(O-E)²/E = 0.77	3.56
Other	(O-E)²/E = 0.24	(O-E)²/E = 0.14	(O-E)²/E = 0.01	0.39
Total	2.88	0.58	0.89	Σ = 4.35

Step 6: degrees of freedom, confidence level and critical Chi-Square values:
(See example 1 for details)
df = (# of nationalities - 1) x (# of neighborhoods - 1) = (3-1)x(3-1) = 4
This sample therefore has 4 degrees of freedom
The critical values of Chi-Square are as follows (see online table):

• For 99.9% certainty and df = 4: Chi-Square critical = 18.5
• For 99% certainty and df = 4, Chi-Square critical = 13.3
• For 95% certainty and df = 4, Chi-Square critical = 9.5

Step 7: Interpret your results
Let's compare the Chi-Square coefficient of the sample (step 5) with the critical value of Chi-Square (step 6) (see step 7 of example 1 for more information): Chi-Square = 4.35 < 9.5 (critical value for the lowest confidence level). The confidence level is NOT met and the test is inconclusive: we cannot reject the Null Hypothesis, which means that the two attributes (nationality and preferred neighborhood to stay) are probably NOT related. Based on this sample, the nationality of tourists does not seem correlated with where they choose to stay in San Francisco. Since most tourist appear to choose to stay in the Tenderloin over the Marina regardless of their nationality, further analysis may try to explain this phenomenon: more hotels? Accessibility? Advertisement?