A Note Regarding the Berkeley Survey Research Center's Paper on the Effects of Electronic Voting in Florida During the 2004 Presidential Election

By Jason Lenderman

A friend of mine asked me to take a look at a report put out by Professor Hout and his associates at the UC Berkeley Survey Research Group regarding the effects of electronic voting in Florida during the 2004 Presidential elections. Though my analysis is far from comprehensive I think it does raise some  interesting questions and therefore might be of interest to people who follow this sort of thing. For the rest of the note I will assume that the reader is already familiar with Professor Hout's report titled "The Effect of Electronic Voting Machines on Change in Support for Bush in the 2004 Florida Elections." All analysis below is done using the data compiled by the Berkeley Survey Research Group and is available on their website.

To understand what if any impact touchscreen voting had on the vote counts in Florida it seems a first reasonable step would be to compare the change in percent of votes for President Bush from 2000 to 2004 for counties using touchscreen and counties not using touchscreen. Below are basic summary statistics for the difference in the percent of votes for President Bush (i.e. b_change)  for the two types of counties.

touchscreen='true'

Min.	1Q	Median	Mean	3Q	Max.
-0.01301	0.01640	0.02081	0.02424	0.03182	0.07364

touchscreen='false'

Min.	1Q	Median	Mean	3Q	Max.
-0.02957	0.02279	0.04004	0.04071	0.06571	0.10710

From this we can see that in general that President Bush's performance actually improved more from 2000 to 2004 in counties that did not use touchscreen when compared to counties that did use touchscreen. At first this would seem to contradict Professor Hout's report, however, in his analysis he includes a number of other factors, in addition to the etouch indicator variable, that could be useful in explaining the change in support of President Bush from 2000 to 2004. It is only after adding these additional factors that the impact of touchscreen voting appears to benefit President Bush. Below I reproduce Professor Hout's OLS analysis of Model 2.

Model 2 (Same as BSRC's Model 2) Call: lm(formula = b_change ~ etouch + income + hispanic + b00pc +     b00pc_sq + size + b00pc_e + b00pcsq_e + v_change + d96pc) Residuals: Min 1Q Median 3Q Max -0.0595442 -0.0117843 0.0008206 0.0109508 0.0491037 Coefficients: Variable Coeff estimate Std Error t value Pr(>|t|) (Intercept) -2.130e-01 9.423e-02 -2.260 0.02770 * etouch 4.166e-01 1.495e-01 2.786 0.00726 ** income -8.173e-07 7.557e-07 -1.081 0.28413 hispanic -5.256e-02 3.075e-02 -1.709 0.09295 . b00pc 1.028e+00 3.223e-01 3.190 0.00233 ** b00pc_sq -6.641e-01 2.809e-01 -2.365 0.02154 * size -3.929e-08 6.689e-08 -0.587 0.55928 b00pc_e -1.284e+00 5.554e-01 -2.311 0.02453 * b00pcsq_e 9.380e-01 5.187e-01 1.808 0.07594 . v_change -2.674e-11 3.002e-07 -8.9e-05 0.99993 d96pc -1.521e-01 1.170e-01 -1.300 0.19896 Signif. codes:  0 `***' 0.001 `**' 0.01 `*' 0.05 `.' 0.1 ` ' 1 Residual standard error: 0.02137 on 56 degrees of freedom Multiple R-Squared: 0.5375,     Adjusted R-squared: 0.4549 F-statistic: 6.507 on 10 and 56 DF,  p-value: 1.426e-06

From the above t-tests it is clear that the deletion of any one of the three touchscreen related variables from Model 2 has a statistically significant impact on the performance of the model in predicting the change of support for President Bush from 2000 to 2004. However, the interaction variables (b00pc_e and b00pcsq_e) seem a bit unintuitive and it would therefore be of interest to see what we would get if we deleted them from Model 2. I show this analysis in table below:

Model 2.1 (Same as BSRC's Model 2 but with b00pc_e and b00pcsq_e removed) Call: lm(formula = b_change ~ etouch + income + hispanic + b00pc +     b00pc_sq + size + v_change + d96pc) Residuals: Min 1Q Median 3Q Max -0.0571328 -0.0136158 -0.0001053 0.0139674 0.0547429 Coefficients: Variable Coeff estimate Std Error t value Pr(>|t|) (Intercept) -2.145e-02 8.631e-02 -0.248 0.8046 etouch -2.042e-03 8.353e-03 -0.244 0.8078 income -1.038e-06 8.043e-07 -1.290 0.2021 hispanic -6.614e-02 3.369e-02 -1.963 0.0545 . b00pc 4.634e-01 3.073e-01 1.508 0.1370 b00pc_sq -1.370e-01 2.560e-01 -0.535 0.5946 size 5.284e-08 6.840e-08 0.772 0.4430 v_change -1.662e-07 3.275e-07 -0.507 0.6138 d96pc -2.305e-01 1.253e-01 -1.839 0.0710 . Signif. codes:  0 `***' 0.001 `**' 0.01 `*' 0.05 `.' 0.1 ` ' 1 Residual standard error: 0.02359 on 58 degrees of freedom Multiple R-Squared: 0.4158,     Adjusted R-squared: 0.3353 F-statistic: 5.161 on 8 and 58 DF,  p-value: 6.977e-05

Notice that we lost some predictive power (as we would expect from t-statistics of the deleted variables) as indicated by the drop in the R^2 from 0.537 to 0.4158, but also notice that the etouch variable is no longer statistically significant (p-value=0.80776) and indicates that it is not adding anything to Model 2.1 in terms of predictive power. Now one may argue that the touchscreen interaction variables, in spite of their somewhat unintuitive nature, should not be deleted from the model because they are obviously adding to the ability of the model to describe the dependent variable. Can we find another interaction variable, perhaps more natural than the touchscreen interactions, that captures all the information of the touchscreen variables? Below I show the OLS analysis of a model with the same set of variables as Model 2 but with an additional interaction (i.e. b00pc_vchange) between support for President Bush in 2000 and change in voter turnout from 2000 to 2004:

Model 2.2 (Same as Model 2 but with b00pc_vchange added) Call: lm(formula = b_change ~ etouch + income + hispanic + b00pc +     b00pc_sq + size + b00pc_e + b00pcsq_e + v_change + d96pc +     b00pc_vchange) Residuals: Min 1Q Median 3Q Max -0.0564784 -0.0112904 0.0008757 0.0099263 0.0432258 Coefficients: Variable Estimate Std_Error t_value Pr(>|t|) (Intercept) -2.768e-01 8.861e-02 -3.124 0.002849 ** etouch -2.032e-01 2.295e-01 -0.885 0.379810 income -3.839e-07 7.060e-07 -0.544 0.588760 hispanic -6.075e-02 2.835e-02 -2.143 0.036571 * b00pc 1.131e+00 2.976e-01 3.799 0.000365 *** b00pc_sq -6.724e-01 2.580e-01 -2.606 0.011760 * size -7.191e-08 6.220e-08 -1.156 0.252644 b00pc_e 8.110e-01 8.040e-01 1.009 0.317550 b00pcsq_e -8.035e-01 7.028e-01 -1.143 0.257889 v_change 3.402e-06 1.046e-06 3.252 0.001962 ** d96pc -1.611e-01 1.075e-01 -1.498 0.139742 b00pc_vchange -6.127e-06 1.818e-06 -3.371 0.001376 ** Signif. codes:  0 `***' 0.001 `**' 0.01 `*' 0.05 `.' 0.1 ` ' 1 Residual standard error: 0.01963 on 55 degrees of freedom Multiple R-Squared: 0.6167,     Adjusted R-squared:  0.54 F-statistic: 8.043 on 11 and 55 DF,  p-value: 3.891e-08

First note that Model 2.2 has a better fit than both Model 2 and Model 2.1 as evidenced by the R^2's (compare 0.6167 to 0.5375 and 0.4158 respectively.) Also notice that even though the t-test for the b00pc_vchange is very significant (p-value=0.001376), none of the touchscreen related variables show statistical significance in terms of their t-tests, which indicates that deleting anyone of these variables individually from Model 2.2 would not have a significant negative impact on the predictive power of the model. Let's now fit a model with all three touchscreen related variables deleted to see how their simultaneous removal impacts the fit of the model. The OLS analysis of this model is in the table below:

Model 2.3 (same as Model 2.2 but with etouch, b00pc_e and b00pcsq_e removed) Call: lm(formula = b_change ~ income + hispanic + b00pc + b00pc_sq +     size + v_change + d96pc + b00pc_vchange) Residuals: Min 1Q Median 3Q Max -0.0587303 -0.0101167 -0.0006346 0.0100548 0.0453931 Coefficients: Variable Estimate Std_Error t_value Pr(>|t|) (Intercept) -2.671e-01 8.623e-02 -3.098 0.003004 ** income -8.388e-07 6.552e-07 -1.280 0.205567 hispanic -6.125e-02 2.803e-02 -2.185 0.032961 * b00pc 1.141e+00 2.875e-01 3.970 0.000200 *** b00pc_sq -7.174e-01 2.409e-01 -2.978 0.004230 ** size -6.471e-08 5.822e-08 -1.111 0.270941 v_change 2.921e-06 6.614e-07 4.417 4.44e-05 *** d96pc -1.349e-01 1.059e-01 -1.274 0.207874 b00pc_vchange -5.241e-06 1.033e-06 -5.073 4.31e-06 *** Signif. codes:  0 `***' 0.001 `**' 0.01 `*' 0.05 `.' 0.1 ` ' 1 Residual standard error: 0.01965 on 58 degrees of freedom Multiple R-Squared: 0.595,      Adjusted R-squared: 0.5391 F-statistic: 10.65 on 8 and 58 DF,  p-value: 4.631e-09

Here see that Model 2.3 is outperforming Model 2 as indicated by the R^2's (compare 0.595 to 0.537.) Looking to the adjusted R^2's (an estimate similar to R^2 but corrected for the number of parameters) we see the comparision is even more favorable for Model 2.3 (compare 0.5391 to 0.4549.) Furthermore, the adjusted R^2's of Model 2.3 and Model 2.2 are almost identical (compare 0.5391 to 0.54) which indicates that the deletion of the touchscreen related variables has no negative impact on the predictive power of the model. We can confirm this in a more rigorous manner by computing the F-statistic which is 1.037 and has a p-value of 0.3835 indicating no statistically significant difference in the quality of the fit of Model 2.2 and Model 2.3. All this strongly indicates that the single interaction between the support for President Bush in 2000 and the change in voter turnout from 2000 to 2004 (i.e. b00pc_vchange) contains all the information (and then some) about the dependent variable as is contained in the three touchscreen related variables. In short, b00pc_vchange makes the touchscreen variables superfluous.

The BCRG working paper pointed out some seemingly unusual behavior in large Florida counties, in particular Palm Beach and Broward, where the actual increase in support for President Bush was much larger than what was predicted by their model (i.e. Model 2) Below we reproduce the analysis of Professor Hout by using Model 2 to determine how each of the counties that used touchscreen voting might have behaved had touchscreen not been used by setting etouch, b00pc_e and b00pcsq_c to 0. We can then compare this to the actual change in support the for incumbent in these counties. The table below shows the difference between the acutal and predicted (i.e. actual change in support minus predicted change in support assuming no touchscreen) for each of the counties using touchscreen.

County	Difference between actual and predicted (with no touchscreen) from Model 2
Collier	-0.037795347
Nassau	-0.021095037
Indian_River	-0.018264850
Lee	-0.018041505
Lake	-0.013199690
Martin	-0.009588123
Sumter	-0.006414797
Charlotte	0.011579074
Hillsborough	0.012229817
Pasco	0.012717679
Pinellas	0.015816356
Miami-Dade	0.020294309
St_Lucie	0.021347976
Palm_Beach_County	0.076411329
Broward	0.107700283

As expected we see an unsually large difference between the actual change and the predicted change in the cases of Palm Beach and Broward. Aside from these two counties everything else seems to be fairly well-behaved.  The mean difference between actual and predicted is 0.01024650 and the standard deviation of the differences is 0.03804717 (note that the std. dev. of the differences of all counties is 0.02617977, and 0.02157821 for non-touchscreen counties.) The fact that the mean difference is substantially larger than 0 seems to indicate an unusual advantage for President Bush in the touchscreen counties. Now lets carry out the same analysis for Model 2.3, by comparing its predictions in touchscreen counties with the actual changes in support for the incumbent. Note that in this case, unlike in the case of Model 2, we do not need to set the touchscreen related variables to 0 since no touchscreen variables are used in Model 2.3. In any case, the table of residuals follows:

County	Difference between actual and predicted (with no touchscreen) from Model 2.3
Indian_River	-0.0202519907
Collier	-0.0202094053
Charlotte	-0.0166581991
Nassau	-0.0144548304
Palm_Beach_County	-0.0107724532
Martin	-0.0054572002
Lake	-0.0042794333
Lee	-0.0037735232
Broward	-0.0022587485
Miami-Dade	0.0006489641
Pinellas	0.0029210902
Hillsborough	0.0046314977
St_Lucie	0.0091020704
Pasco	0.0151637158
Sumter	0.0218939045

As we can see, using Model 2.3, the predictions are not only more accurate (as was shown in the earlier OLS analysis) but also better behaved - the seeming anomalies in Palm Beach and Broward that were seen using Model 2 are gone. For comparison with the results of Model 2, the mean of the differences between the acutal and the predicted for Model 2.3 is -0.002916969 and the standard deviation of the differences is 0.01248824 (note that the std. dev. of the difference between actual and predicted over all counties is 0.01841657.)  We can therefore conclude that according to Model 2.3 (which from the perspective of goodness of fit and parsimony is preferable to Professor Hout's Model 2) the effect of touchscreen voting is a wash, with President Bush showing no unusual advantage or disadvantage in these counties.

In conclusion, we have found that the interaction between support for President Bush in 2000 and change in voter turnout from 2000 to 2004 (b00pc_vchange) makes the three touchscreen variables (etouch, b00pc_e and b00pcsq_e) of Professor Hout's Model 2 superfluous. By using b00pc_vchange as in Model 2.3 we get a better fit (with fewer and arguably more intuitive variables) than can be achieved by using the touchscreen related variables. Moreover, the touchscreen variables have nothing to add over and above b00pc_vchange in terms of  extra predictive power. For these reasons I have to conclude that the statistical evidence of a touchscreen voting effect, as presented in the Berkeley study, is at best questionable.