The client is a leader in IP networking technology, with a strong track record in developing and deploying next-generation carrier-grade Session Border Controllers (SBC), pushing the envelope in an all IP paradigm. The client has a global presence with its SBC boxes used globally by leading Telecom Operators.
Network operators (the client’s customers) were losing a large amount of money because of fraud. The client did not have sufficient infrastructure to support running complex clustering algorithms to identify and detect fraud.To build a system that allows storage of large amounts of call data on a single SBC box and be linearly scaled to accommodate data for more than a year. The system should leverage data to generate fraud patterns, and identify any fraudulent activity. For this, the system needs to provide real-time reports and dashboards for monitoring fraud and other key quality measures. However, additional challenges include
Large volume and data velocity (over 10,000 calls per second)Complex application data (flat files, Google protocol buffers, nested data structure)Limited Resource on SBC Box, scalable with node addition
A big data platform was needed to fulfill the ambitious requirements of the clientApache Spark with Parquet columnar storage was selected for compressionElephant Bird and Java were used for Google Protobuf ProcessingKVM Virtualization of SBC Server to run multiple nodes with redundancyReal-time scaling with addition of physical nodesUnsupervised clustering & supervised classification for pattern recognitionStatistical parameters such as average length of call, average number of calls per month and average delays in bill paymentReal time fraud detection and alertingThe client is now able to retain and report data for 1 year on the same box instead of 1 month’s data.The client was able to monetize the big data platform by adding advanced reporting features.Fraud management was introduced in the product for identifying pre-defined fraud patterns in real time and report to the operator.