The Practitioner Panel Survey is an important but frequently misinterpreted data set, and has a long a varied history going back to the early days of the FSA. it’s good to see this is now being done jointly by the panel and the regulator, as it’s sometimes been a source of semi-public discord between them, but there will still be a set of difficult issues to be navigated. 

Some of these are technical (what's the best way to cut the results?), some are conceptual (what are we trying to find out?) and others are (small p) political. Inevitably these aren’t mutually exclusive. Here’s a small selection:

1. Depending on how you cut the results (e.g. by sector or size of firm), the picture can appear very different. Whether a firm is in the FCA’s fixed portfolio (the largest firms, who have a named supervisor) or the flexible one, can heavily influence its impression of the regulator. Likewise, whether you are in a sector that has gone through a major regulatory change (e.g. MiFID II) is likely to affect your view.

2. Sample size matters, particularly when there is an element of self-selection in those who reply. By these standards, the 22% of firms who responded to the last survey looks low.

3. There is a natural tension between wanting to find out how specific events/regulatory initiatives are viewed by firms, and on the other hand looking for longer term trends in the underpinning attitudes of the regulated population. Distinguishing between the two is hard in any case and the reputational impact of a specific event – whether positive or negative – is often difficult to gauge at the time.

4. Scope change also has a huge impact. The most recent instance is the FCA taking responsibility for consumer credit, but at the time the impact of the advent of Mortgage and General Insurance regulation in 2005 was probably as great. Taking proper account of the views of new firms (often small and in some cases new to regulation) is a major challenge.

5. Given the above, it is easy to read too much into the results of a single survey and the changes from last time, there are simply too many variables. But the temptations to do so, both positive and negative, can be huge. The FCA’s Mission is an obvious example, and both the regulator and the Panel will inevitably look hard for signs it is making a difference when the reality is it’s too soon to tell.

6. Finally, there is, I think, a legitimate question about whether a regulator can be too popular. If, put simply, part of the role is to prevent/deter regulated firms from doing some of the things they want to do, then high approval scores carry dangers for firms and regulator alike.

Having moved on from working for the regulator after so long, it’s also obvious to me that there is a good deal of “lost in translation” around how firms and regulator communicate. The Practitioner Panel Survey doesn’t solve this problem but can provide a way in to greater understanding.