SUMMARY
Much has been made of the Australian Competition & Consumer Commission analysis of the Western Australian FuelWatch scheme. This paper investigates the ACCC analysis published last December (Appendix S) and comments briefly on the additional analysis the ACCC released on May 29, 2008.
The ACCC data has not been released to the public, nor have they been very clear as what analysis has been done. The ACCC has been vague in reporting their econometric techniques and have simply published tables, expecting the public to accept their analysis at face value.
The ACCC did not investigate the entry of Coles into the WA market in their analysis. The analysis shown below does do that, and finds that the "Coles effect" completely dominates the FuelWatch effect. Once the Coles effect is included in the analysis it is clear that FuelWatch has had no effect on average monthly prices in the WA market.
The analysis below consists of comparing average monthly prices before and after FuelWatch started up and before and after Coles entered the WA market. That very basic analysis is then confirmed using basic and standard (ordinary least squares) regression analysis.
The original ACCC analysis is found to be very sensitive to starting and end points, consequently the ACCC analysis can not be described as being "robust".
On May 29, 2008 the ACCC released new and apparently more sophisticated analysis, but do not say what type of econometric analysis they performed. Indeed the analysis did not even report standard diagnostic statistics such as standard errors or p-values. This makes it impossible to analyse the new results. It is important to note, however, that in the later analysis the ACCC argues that "there is no evidence that the introduction of Fuelwatch in Western Australia led to any increase in prices", while originally they argued that the evidence showed that the relative price margins were lower after FuelWatch was introduced than before.
- The ACCC analysis is not convincing.
- The ACCC analysis remains unconvincing after 29 May -- the new analysis is even vaguer than the original analysis released in December 2007.
- The ACCC did not test for a "Coles effect" in their December 2007 analysis.
- The Coles effect totally dominates the FuelWatch effect.
- Based on the analysis shown below it appears that FuelWatch has had no effect on the average monthly price differential in WA relative to the eastern states.
- The ACCC claim to now test for a Coles effect, but do not say what test they have actually performed.
INTRODUCTION.
The FuelWatch scheme was introduced in Western Australia in January 2001. The purpose of the scheme is to provide certainty to consumers as to petrol prices for a fixed period of time. In practice, service stations are required to notify FuelWatch of their prices for the next day. On the following day, beginning at 6am the service station prices are fixed for 24 hours.
In December 2007, the Australian Competition & Consumer Commission (ACCC) released its report into petrol pricing in Australia. This report included an Appendix (Appendix S) that contained an econometric analysis of the FuelWatch scheme. That econometric analysis has been widely cited as demonstrating that the FuelWatch scheme has led to lower prices in Western Australia. This paper provides a critique of that analysis and shows that the ACCC analysis is not robust and does not demonstrate what it purports to demonstrate.
On the 29 May 2008 the ACCC released a document that purported to provide details of "further FuelWatch econometric analysis". That document, however, moves the goalposts by claiming that "there is no evidence that the introduction of Fuelwatch in Western Australia led to any increase in prices". The new ACCC analysis that the introduction of FuelWatch "appears to have resulted in a small price decrease overall".
THE ORIGINAL ACCC ANALYSIS.
The ACCC collected weekly, monthly and weekly minimum data for the period 1 August 1998 to 8 June 2007. They then calculated the following Price Margin measure
Price Margin = (Retail price – lagged Mogas95 price – net taxes – fuel quality premium) Perth
less
(Retail price – lagged Mogas95 price – net taxes – fuel quality premium) average of eastern capitals (1)
The ACCC defend this measure on the basis that it removes factors that are beyond the control of FuelWatch. The lagged Mosgas95 price is the base supply price of petrol and is lagged one week. It is difficult to understand why this figure has been subtracted from the retail price as I imagine it would be constant across Australia. Unfortunately, the ACCC analysis gives no indication as to whether this figure does vary across the various states. Similarly it is not clear whether the net taxes figure varies across states. The mandated fuel quality does vary across states, but the ACCC analysis gives no indication as to what those figures or variations might be. In other words, the measure of interest is not transparent. The ACCC does not provide any summary data.
The ACCC then undertakes a "unit-root test" to ensure the measure is stationary. This is important for technical econometric reasons. The ACCC analysis then investigates whether the data exhibit a structural change after the introduction of FuelWatch. It appears that the ACCC estimated the following equation:
Price Margint = α + βFWt + εt (2)
Where α = constant representing the average Price Margin before the FuelWatch scheme was introduced, β = the average impact of the FuelWatch scheme, FWt is a dummy variable = 1 after 2 January 2001 and = 0 before 2 January 2001 and εt = an error term.
The ACCC estimate three versions of the equation, one for each of the three time series versions of Price Margin. They report the results in their Table S2 (reproduced below).
Table S2 Structured break test 3 for relative price margin, cpl,
August 1998 to June 2007
| Series | Average (August 1998 to December 2000) | Average (January 2001 to June 2007) | ||
| Weekly average | 0.83 | (0.002) | -1.92 | (0.000) |
| Monthly average | 0.88 | (0.001) | -1.86 | (0.000) |
| Weekly minimum | 0.30 | (0.277) | -0.90 | (0.003) |
3 Coefficient given with p-value in brackets.
Diagnostic testing indicated serial correlation so Newey West standard errors used.
Source: ACCC estimates
To understand this table, look at the Monthly average row. The number 0.88 indicates that there was, on average a 0.88 cent per litre (cpl) difference between the Perth net price and the average of the eastern capitals net price before FuelWatch was introduced. This figure corresponds to the α-term in the equation (2). The number in parenthesis (0.001) indicates that the 0.88cpl difference is statistically significantly different from zero. The figure -1.86 represents the impact FuelWatch had on the Price Margin; this is the β-term in equation (2). This implies that the Perth net price fell, on average, by 1.86cpl relative to the average of the eastern capitals following the introduction of FuelWatch. The number in parenthesis (0.000) indicates that the -1.86cpl difference is statistically significantly different from zero.
This analysis is consistent with the argument that the FuelWatch scheme lead to lower prices following its introduction in 2001. The ACCC does, however, discuss some caveats to the analysis. For example, the impact of the fuel quality premium is considered. If we assume that the Mosgas95 price is equal across Australia and the net taxes on fuel are equal across Australia, the equation (1) can be reduced as follows:
Price margin = (Retail price Perth – Retail price Average of eastern capitals)
less
(Fuel quality premium Perth + Fuel quality premium Average of eastern capitals)
If the relative retail price premium is reasonably constant over the whole time period, but the relative fuel quality premium increases over time, we might expect the β-term in (2) to be negative. The ACCC claims that the difference in fuel quality premia has decreased over time. Presumably the eastern states have increased the quality of their fuels and WA has not reduced the quality of its fuels. In other words, the relative quality fuel premium could be explaining the results. The ACCC claims to have investigated whether this is driving the results, but does not report that analysis other than to say the overall result is robust to the exclusion of the fuel quality premium.
AN ALTERNATE ANALYSIS.
The ACCC argue that, "Of potentially greater concern is the possibility that something else entirely has driven the improvement in the relative price margin." That is always a possibility. The ACCC, however, do not investigate the most obvious other factor -- the entry of Coles into the WA market in March 2004. Using monthly data collected from the Australian Automobile Association I investigate that possibility.
I calculate the following equation:
Relative Pricet = α + β1FWt + β2Colest + εt (3)
Where Relative Pricet = Average Price Perth – Average Price Eastern Capitals in month t, α = a constant representing the average Relative Price before the FuelWatch scheme was introduced, β1 = the average impact of the FuelWatch scheme, FWt is a dummy variable = 1 after 2 January 2001 and = 0 before 2 January 2001, β2 = the average impact of the entry of Coles, Colest is a dummy variable = 1 after March 2004 and = 0 before March 2004 and εt = an error term.
I calculate two measures for Relative Price. The first measure P1 is the Perth Average monthly price less the average of the monthly averages for Adelaide, Darwin, Melbourne, Hobart, Sydney and Brisbane. The second measure P2 is the Perth Average monthly price less the average of the monthly averages for Adelaide, Melbourne, Sydney and Brisbane; this corresponds to the eastern capitals that the ACCC used in their analysis. Consistent with the ACCC analysis, I use the time period August 1998 to June 2007 for the empirical analysis. The graph shows the time series of the data and table one shows some summary statistics.
Looking at the graph, it is not clear that the relative price responded much to the introduction of FuelWatch in January 2001. The large decline in the relative price occurred after December 2003 (when Woolworths entered into the WA market) and continued after Coles entered the WA market in March 2004.
Table One: Summary Statistics for Relative Price.
| P1 | P2 | |
| Average Before FuelWatch | -0.50 | 2.60 |
| Average After FuelWatch | -1.33 | 1.57 |
| (0.0244) | (0.0041) | |
| Average After Coles | -2.51 | 0.45 |
| (0.0000) | (0.0000) | |
| Average After FuelWatch before Coles | -0.03 | 2.84 |
| (0.0539) | (0.1905) |
The numbers in parenthesis are p-values from a two-sided t-test for equality of averages.
The analysis in table one is initially consistent with the ACCC analysis. Simply looking at the before and after FuelWatch averages and the associated t-test p-values, it appears that prices in WA did fall after the introduction of FuelWatch. The price effect, however, is far stronger after the introduction of Coles. Finally, I look at the relative price after FuelWatch was introduced but before Coles entered the WA market. The analysis suggests that prices rose by a statistically significant amount, on average, for P1, but not by a statistically significant amount for P2.
In order to establish the base case, I first estimate equation (3) without the Coles variable. Results are shown in table two.
Table Two: FuelWatch structural break test for relative prices
(August 1998 – June 2007).
| Constant | FuelWatch | Adj-R2 | |||
| P1 | -0.5107 | (0.0434) | -0.8202 | (0.0813) | 0.0375 |
| P2 | 2.5931 | (0.0000) | -1.0328 | (0.0290) | 0.0675 |
Numbers in parenthesis are p-values.
Standard errors are Newey-West corrected.
P1 compares average WA prices to all other states,
P2 compares average WA prices to SA, QLD, Vic and NSW.
The results are broadly consistent with the ACCC analysis. P2 corresponds more closely to the ACCC analysis as it compares the average WA price to the same four eastern states as the ACCC analysis. P1 compares the average WA price to all the other states. It is interesting to note that the effect of FuelWatch is much weaker when Tasmania and the Northern Territory are added to the mix. Indeed, the price reduction is less than 1cpl, and only statistically significant at the 10 percent level. The P2 analysis, however, has FuelWatch delivering a saving of 1.03 cpl on average, and is statistically significantly different from zero. The ACCC did not report adjusted R2 measures in their analysis -- but it can be seen that they are very low.
I now include the Coles variable into the analysis. Results are shown in table three.
Table Three: FuelWatch and Coles structural break test for relative prices
(August 1998 – June 2007).
| Constant | FuelWatch | Coles | Adj-R2 | ||||
| P1 | -0.5107 | (0.0444) | 0.4814 | (0.2422) | -2.5382 | (0.0000) | 0.4478 |
| P2 | 2.5931 | (0.0000) | 0.2411 | (0.5413) | -2.4842 | (0.0000) | 0.4734 |
Numbers in parenthesis are p-values.
Standard errors are Newey-West corrected.
P1 compares average WA prices to all other states,
P2 compares average WA prices to SA, QLD, Vic and NSW.
The results are very different from the ACCC analysis. The dummy variable associated with FuelWatch is now not statistically significant. The Coles variable is highly statistically significant and indicates that greater competition in the form of Coles entering the market caused the relative price of fuel to fall by about 2.5cpl. In addition the adjusted R2 are now much higher than before.
In table four I estimate the equation without the FuelWatch variable.
Table Four: Coles structural break test for relative prices
(August 1998 – June 2007).
| Constant | Coles | Adj-R2 | |||
| P1 | -0.2376 | (0.3172) | -2.3299 | (0.0000) | 0.4404 |
| P2 | 2.7299 | (0.0000) | -2.3799 | (0.0000) | 0.4752 |
Numbers in parenthesis are p-values.
Standard errors are Newey-West corrected.
P1 compares average WA prices to all other states,
P2 compares average WA prices to SA, QLD, Vic and NSW.
The impact Coles had on relative prices is now slightly smaller than before at about 2.3cpl, while the adjusted R2 is still very high. This confirms, to my mind, that the petrol price saving in WA is due to the entry of Coles into the market in 2004 and not the introduction of the FuelWatch scheme in 2001. The ACCC analysis does not consider this possibility at all and is, at least, fundamentally incomplete and flawed as a consequence.
OTHER CONCERNS.
As one of the ACCC caveats they write, "Different timeframes could conceivably give different results". That is very true. The ACCC regression analysis begins in August 1998. They claim that they cannot go before that date given the price deregulation that had occurred. That is a plausible argument. But what happens if they were to begin their analysis using later data? In table five I show what happens to the FuelWatch term as the starting period moves later.
Table Five: Starting Point Sensitivity Analysis.
| FuelWatch | ||||
| Start Date | P1 | P2 | ||
| Aug-98 | -0.69 | (0.0813) | -1.03 | (0.0290) |
| Sep-98 | -0.75 | (0.0721) | -1.05 | (0.0287) |
| Oct-98 | -0.80 | (0.0816) | -1.02 | (0.0349) |
| Nov-98 | -0.84 | (0.0979) | -0.99 | (0.0414) |
| Dec-98 | -0.86 | (0.1222) | -0.94 | (0.0530) |
| Jan-99 | -0.82 | (0.1570) | -0.86 | (0.0807) |
Table shows the β-term from equation (2),
substituting Relative Price for Price Margin,
and the associated Newey-West adjusted p-value.
As can be seen, the benefit of the FuelWatch scheme declines as the starting period changes. By December 1998, the benefit using the P1 measure is no longer statistically significant, while the P2 measure (the one chosen by the ACCC) benefit remains significant at the 10 percent level of confidence, but is much smaller than before.
I also investigate what would have happened if the FuelWatch scheme had been evaluated at the end of 2003 i.e. before Coles entered the market in March 2004 (and at the same time Woolworths entered the market in December 2003). Results are shown in table six. As can be seen the FuelWatch scheme would have been declared to have had no effect at all on relative prices. While the FuelWatch terms are positive, they are not statistically significantly different from zero.
Table Six: End Point Sensitivity Analysis.
| Constant | FuelWatch | Adj-R2 | |||
| P1 | -0.5107 | (0.0370) | 0.5141 | (0.2028) | 0.0305 |
| P2 | 2.5931 | (0.0000) | 0.2680 | (0.4898) | 0.0015 |
Table shows the β2-term from equation (3) and the associated Newey-West adjusted p-value.
Data ends at December 2003.
THE MAY 29 ACCC ANALYSIS.
The ACCC claim to have undertaken an "endogenous selection of structural break points". Unfortunately, it is not at all clear what this may mean; the ACCC do not even state which econometric test they have performed. They purport to show results but do not indicate whether these results are statistically significant. Without any knowledge of the test being performed or any tests of significance it is impossible to evaluate this effort.
No comments:
Post a Comment