Why Y'all Should Never Purpose The Hodrick-Prescott Filter

One really mutual trace of piece of work inwards finance together with economic science is to calculate the underlying vogue of a fourth dimension series. This is a well-known job inwards communication systems, together with it is accomplished yesteryear designing a low-pass filter: a filter that eliminates high-frequency components of an input. For hard-to-understand reasons, some economists job the Hodrick-Prescott Filter (the "HP Filter") equally a low-pass filter. Unfortunately, the HP Filter violates several principles of filter design, together with generates misleading output. As a result, it should never hold upward used. Although this theme sounds fairly technical, problems tin hold upward easily illustrated graphically. Even if you lot are non interested inwards filtering serial yourself, these problems must hold upward kept inwards hear when looking at economists' inquiry if it is based upon the job of this filter. The conclusions may hold upward based on defects created yesteryear the filtering technique.

Background

The implementation of the HP Filter is quite complex, together with my guess is that this complexity gives the technique an aura of sophistication. Since you lot should never job the filter, at that topographic point is no argue to human face at the mathematics behind it. But if you lot wish, the HP filter is described inwards this R linguistic communication documentation (the mFilter package, I used this packet to generate the results herein).

The nautical chart higher upward shows a typical job of the HP Filter - using it to calculate "trend" (real) GDP. The summit panel shows the master copy serial together with the filter output, which is referred to equally the vogue inside the economic science literature. The vogue is a prissy smoothen trace that fits through the gross domestic product series. (Please complaint that I am using this equally an representative that provides a uncomplicated intuition for to the problems amongst the HP filter. As I complaint below, although the HP filter was used for this purpose historically, most serious attempts to calculate potential gross domestic product at nowadays job different techniques.)

(On a technical note, I calculated the serial equally follows. I start transformed gross domestic product yesteryear taking the logarithm of its value. This converts steady exponential increase to a directly line. I hence calculate the vogue of the log GDP, either using the HP filter or an alternative technique. Finally, I calculate the vogue for gross domestic product yesteryear taking the exponential of that vogue series. All of my calculations job the smoothing parameter value of 1600, which is the same value that everyone else uses for quarterly data. Why 1600, together with non 100? There is a long explanation (which may or may non hold upward convincing); Professor Karl Whelan has lecture notes amongst a summary explanation. [UPDATE: Please complaint that the master copy explanation of the "1600" parameter choice was sarcastic together with completely unfair. Apparently, I am inwards a to a greater extent than generous mood today, together with updated the text. I would similar to give thank you lot the commenter 'elsurexiste' for providing the reference together with rapping my knuckles almost my remark.]

The bottom panel shows the divergence of gross domestic product from the vogue calculated yesteryear the HP filter. It is shown equally a pct of GDP.  Interestingly enough, it shows that U.S.A. existent gross domestic product is 1% higher upward trend, which should hold upward scary if you lot are a bond bull - together with if the HP filter was reliable (I speak over this farther below).

Since the filtered serial I calculate are quite some each other inwards degree terms, I volition at nowadays demonstrate filter outputs yesteryear the divergence inwards vogue that they generate.

As an aside, many people job a divergence of gross domestic product from some vogue equally a mensurate of the "output gap". I would signal out that the 2 concepts should hold upward separated, equally nosotros involve to brand strong model assumptions to justify the belief that the output gap should hold upward some vogue GDP. As an example, the relatively depression charge per unit of measurement of increase of gross domestic product since the destination of the Financial Crisis implies that vogue gross domestic product increase is lower, but that does non necessarily hateful that potential gross domestic product increase is low.

Why The HP Filter Fails For "Real-Time" Usage

The most of import job amongst the HP filter tin hold upward demonstrated yesteryear how it behaves nether different scenarios. In the summit panel of the nautical chart above, I demonstrate iii fourth dimension serial of the increase rate: historical information (until 2014q4, inwards black), a scenario of "strong growth" (more accurately, "stronger growth" -  a 3% annual rate, inwards red), together with a scenario of a mild recession (2% contraction over 2 quarters, together with hence returning to 1.5% annualised growth, inwards blue).

The bottom panel shows what tumble out to the HP filter output, depending upon the scenario. The divergence from vogue at the destination of the information laid is revised. If nosotros bring stronger growth, the approximate of existent gross domestic product beingness higher upward vogue at the destination of 2014 is revised away, together with it appears that the economic scheme was exclusively nigh the vogue value. Conversely, if the economic scheme drops into recession, the economic scheme looks similar it was equally far higher upward vogue equally it was inwards 2007.

This happens because the HP filter is non-causal, the value at a signal inwards fourth dimension depends upon futurity values of the fourth dimension series. We tin calculate a non-causal filter output if nosotros bring access to the series' information over all time, but nosotros cannot calculate the output values inwards existent time. For this reason, engineers produce non human face at non-causal filters, other than inwards the initial lectures of a class on systems theory.

Most statistical agencies figured this out some fourth dimension ago, together with they no longer job the HP filter for things similar estimation of vogue GDP. (I bring non actually looked at the novel techniques beingness used, which are varied.) The economists that hold to job the HP filter appear to hold upward theoreticians who are non likewise worried almost econometrics equally good marketplace economists (and bloggers) who throw together analyses fairly quickly.

Within finance, practitioners frequently want to job smoothed serial equally inputs to trading rules. Non-causality straightaway eliminates the HP filter from consideration, equally whatever trading dominion that is based upon futurity prices should generate strong returns inwards backtesting.

Historical Analysis

One could hence combat that perhaps the HP filter tin hold upward used for historical analysis. Since the information that nosotros are looking at is fixed (barring revisions), the non-causality mightiness non hold upward an issue.

The job amongst this trace of declaration is that the HP filter output is unreliable nigh the endpoints of the information set. Unfortunately, nosotros produce non know what "near" means.

The summit panel of the nautical chart higher upward shows how the estimation of the divergence from vogue depends upon the sample you lot choose. The dark trace shows the estimated divergence from vogue when nosotros job a information laid that runs from 1990q1 to 2007q4, together with the carmine trace uses the information laid that ends inwards 2014q4. The endpoint of the shorter sample (end of 2007) shows gross domestic product nigh trend, which is the inherent vogue of the HP filter. This is completely different from the motion-picture demonstrate that is given yesteryear using the larger sample.

The bottom panel shows the analysis using a sensible filter - the centered* moving average (I used 21 periods, or v years together with 1 quarter). The centered moving average is a moving average where the output is fourth dimension shifted yesteryear a "lead" of one-half of the moving average. In this case, the 21-period moving average is the average of a signal inwards time, addition the fourth dimension points that are inside 10 periods on either side. (Since at that topographic point are 10 periods on each side, addition the middle point, at that topographic point are 21 points to hold upward averaged.) Like the HP filter, the centered moving average is non-causal.

The centered moving average is non defined over the entire interval, equally nosotros lose the start together with final 10 points inwards the information set. But where it is defined, it is really some the HP filter value. As I would phrase it, the HP filter is almost equally proficient equally the centered moving average. (I bring made no travail to adjust the filter parameters hence that the outputs are closer together.)

Additionally, the centered moving average tells us where to throw out the information points at the destination of the information - at that topographic point is no output defined for where the results would hold upward sketchier. The HP filter methodology offers no insight into where nosotros involve to lop off the unreliable endpoint data.

H5N1 communication or command organization engineer would object that moving averages (including centered moving averages) are to a greater extent than frequently than non not used inside engineering. They bring some unfortunate tendencies to magnify some high frequency noise. This tin hold upward interpreted equally the "base effect" that you lot encounter inwards the output. (A stupor hits a moving average both when it enters the average, together with when it drops out.) Typically, depression gild transfer functions are used; the most basic of which is known equally the "exponential moving average" inwards economics. If I were edifice a trading rule, I would job these filters instead of a moving average, but for my purposes here, I job the simplest to explicate filter - the moving average. My feeling is that almost all my target audience amongst backgrounds inwards finance or economic science knows what a moving average is; the exponential moving average is unfortunately exotic. (For electrical engineers, the province of affairs is reversed.)  See my primer on "adaptive expectations" for to a greater extent than give-and-take of the exponential moving average filter.

What Do We Do In Real Time?

H5N1 centered moving average is reasonable solution for historical information analysis. For estimates of electrical current values, solutions appear less satisfying. The argue is that if nosotros trammel ourselves to causal filters, nosotros bring an unavoidable fourth dimension lag built into our filter outputs.

The province of affairs for the centered moving average is the easiest to explain. Systems theory texts typically start amongst the non-causal centered moving average, equally it has the cleanest frequency domain representation. We hence add together a fourth dimension lag to larn inwards into a criterion moving average, which is causal. In other words, a moving average is best idea of equally a centered moving average addition a fourth dimension lag.

For other filters, the proof of why a fourth dimension delay is introduced requires the job of frequency domain analysis. There is the equivalent of a conservation police pull that explains why you lot bring a trade-off betwixt filtering high frequency signal components together with the fourth dimension delay inwards the output. The proof of this tin hold upward works life inwards most digital signal processing textbooks.

One apparent exception that is good known inside economic science is the concept of seasonal adjustment. When they seasonally adjust data, statisticians promise to cancel out seasonal dissonance using a model-based approach. Unfortunately, these models volition exclusively piece of work if the seasonal pattern is stable, which is non ever the case. In gild to avoid a bias, seasonal adjustment has to hold upward done inwards a non-causal fashion.  Seasonal adjustment is to a greater extent than of an fine art than a science.

H5N1 More Formal List Of The Defects Of The HP Filter

I volition at nowadays give a to a greater extent than formal brusk critique of problems amongst the HP filter.

1. Non-causality. As discussed above.
2. Unknown amount of invalid points at ends of interval. This job is extremely critical; nosotros produce non know where the information are garbage.
3. Computationally intensive. Wastes resources.
4. Black box implementation.  It is hard to encounter what the properties are of this solution, dissimilar alternatives that bring create clean frequency domain representations.
5. "Magic" input parameter. It is really hard to encounter what the "lambda" parameter represents. (Yes, at that topographic point is a "story" behind it, but it is even hence non especially helpful when compared to sensibly designed digital filters. The flush relies on arbitrarily long inputs, which is just non a feature of ral-world economical series.)

There are no advantages of the HP filter over a centered moving average, together with it has the disadvantages listed above. One tin job a to a greater extent than full general non-causal Finite Impulse Response (FIR) filter to larn a smoother frequency domain reply than the centered moving average if desired, but that would require looking at an undergraduate text inwards digital communications organization inwards gild to sympathize how to shape the filter response.

Footnote:

* "Centered" versus "centred"? The start spelling would hold upward the American preference, whereas the minute English. I follow Canadian spelling patterns, which veer erratically betwixt American together with English linguistic communication usage, which likely causes people on both sides of the Atlantic to believe that I cannot spell. As for "centre", the usage inwards Canada seems to hold upward drifting towards a persuasion that "centre" refers to a building, spell "center" is the signal inwards the middle of something. Therefore, you lot could perchance encounter a judgement similar "The shopping centre is at the middle of the map." Since "centred" looks likewise much "cent-red", I'll stick amongst "centered".

(c) Brian Romanchuk 2015