Service Provider Data Quality in Business Connectivity Market Reviews
By Michael Dargue
Business-to-business (B2B) markets are often more difficult to analyze than consumer markets. These markets tend to be more fragmented, less transparent, have a wider range of customer needs, and often contain specialist products and services that are specific to niche customer segments.
The B2B market for telecoms services has all of these characteristics. Communication Service Providers (CSPs) offer a wide range of specialist services for connecting business locations to the voice network, the Internet, and to one another. Broadly, these services rely on dedicated connections within the network – business connectivity (leased lines).
Where a CSP has coverage, it will use its own network to provide these connections. However, CSPs also serve businesses beyond the reach of their network footprint. In these “off-net” cases, the CSP will lease a connection from an operator that is present in that area.
Because of this on-net / off-net distinction, any assessment of competition and choice of supply needs to consider both the retail market and the network footprints of the wholesale suppliers.
This can be seen in Ofcom’s approach in its Business Connectivity Market Review (BCMR). To understand how much choice business users had when selecting suppliers, Ofcom undertook two spatial analyses:
- A Network Reach analysis, to identify areas where there was alternative network infrastructure to BT. (Ofcom’s metric here being the average number of operators in a postcode sector with a flexibility point [i] within 200 meters of a business site).
- A Service Share analysis, to understand the market shares of different types of leased lines that BT and its competitors supply.
These analyses helped Ofcom to determine whether the market was effectively competitive and to design appropriate remedies. In fact, Ofcom concluded that separate geographic markets existed within the UK, each with its own competitive conditions – a finding that was underpinned by the spatial analysis.
From an analytical perspective, Ofcom’s calculations are quite straightforward and are explained in detail in its consultation document and Cartesian’s audit of Ofcom’s model. The most complex task for Ofcom was to process the large data sets (850,000 records) obtained from the CSPs beforehand, to ensure that the data were reliable and consistent.
The CSP data sets included:
- Inventories of active leased line sales and purchases
- Locations of network flexibility points
- Postal addresses of fiber-connected buildings (customer and network sites)
Anyone who has worked with network operator inventory and CRM systems will tell you that data quality issues are commonplace. The most frequent problems are incomplete records, misspellings, duplicate entries, and billing addresses transposed with service addresses.
These issues are compounded when attempting to reconcile data between multiple service operators, each one having its own ‘standard’ format for customer data records and network inventory. Unless you are familiar with these data quality issues, and have experience in cleansing operator data, the challenge can appear daunting.
In fact, we are aware of at least one other national regulatory agency (NRA) that considered Ofcom’s approach and decided that it would be too difficult to attempt.
We can certainly sympathize with that view. To undertake a manual clean-up of several CRM databases isn’t a task we would wish on anybody. The only way that you can really do this is to apply automated tools in conjunction with a genuine understanding of the data being cleansed.
Cartesian works with operator data on a daily basis and has an established set of in-house and third-party tools for wrangling data in projects such as this. We recommend that any NRA contemplating a similar analysis ensures that it has access to tools similar to the ones we use.
Key considerations for such a toolset include:
- Scale to handle the entire data set
- Ability to detect incomplete and invalid entries
- Ability to efficiently transform data into a consistent format
- Automation of the analytical workflow
- Visualization capabilities (e.g. maps)
Given Cartesian’s experience and expertise, we would be pleased to offer any advice to NRAs seeking to undertake network reach, service share, or similar analyses in the communications markets. Contact us.
As Ofcom found, competitive intensity can vary considerably from one postcode area to the next. It was only through detailed analysis of cleansed CSP data that these local geographic nuances could be properly understood.
Notes:
[i] A flexibility point is a point on an existing network from which a CP can build fiber to connect new customers. These may be housed in buildings, in underground chambers, or where ducts meet at a junction.
