TWPLC - Call Stats

Following on from the success of the Telecoms World PLC website, Telecoms World approached us with an idea to update their online call stats package. Their existing solution from current telecoms providers had little in the way of the customisation and reporting tools that Telecoms World required. Speed was also an issue for the original product with some of the more intensive operations taking several minutes to generate just one report.

The Brief

As a brief, we were given samples of incoming data from the telecom providers and asked to produce a fully-customisable reporting platform. In tailoring these reports to each user’s requirements we couldn’t compromise on performance. Report generation needed to be fast regardless of platform, meaning all interfaces needed to be designed responsively.

Proof Of Concept

We kicked off with a proof of concept to demonstrate the possibilities locked within these large datasets. Using a bare-bones setup and a couple of reports compiled from the sample data, we ran through the results with the Telecoms Worlds team to get their initial thoughts and feedback. With both parties happy, we proceeded to create a full scale web app powered by a REST-like API built on our in-house PHP framework. For the frontend development, we switched to building a JavaScript MV framework on top of Backbone and RequireJS, creating a well-structured, maintainable codebase to extend and reuse wherever we needed.

Developing the frontend this way allowed us to use RequireJS to only load in the specific code as and when it was needed. This saved us bandwidth and speed up initial load times across all devices. Whilst this was useful for all users, it especially benefited those on mobile, one of our target user groups.

This pattern led to an increase in total requests made to the server, but following testing we were happy that the savings from the bandwidth and load speed outweighed the overhead of these requests.

We also made savings to the data transfer by structuring our JSON based API so that we only returned the required data needed to build the reports and data tables.

We used data tables to format the information presented in the charts, updating their own block independently without the need to redraw the entire page. As outlined in the brief, the users have the controls to sort and filter these tables to get the data they require as quickly as possible. All data was made available to download into CSV or Excel format at the click of a button.

Behind The Scenes

The next step was to move from the static sample data to full integration with the live data sources. Data is provided via CSV every fifteen minutes to a nominated FTP server. We created a CLI application to automatically retrieve each CSV and parse the five columns of data into our database. During the parse we take the opportunity to extract further information from the data that enables us to generate more advanced analysis in a few specific client reports.

If the application encounters any errors during the parse then it will wait and try again before sending out a data quality alert to be acted upon.

Data Visualisation

We knew from the outset that the app was to be responsive, which meant that using Flash to generate charts and other data visualisations would be impractical given the limited support on mobile and tablets. Testing led to us adopting the Highcharts library to provide a cross browser Canvas/VML JavaScript generated chart. This allowed rapid building of the charts required for all reports.

In some reporting we combined Highcharts with other JavaScript libraries when required, such as the Google Maps API to create a visual heat-map of each within a user-specified timeframe.

Speed

As was outlined in the brief and in our meetings throughout, speed of the app would be critical. Everyone, including the client, hates to wait for loading pages.

The finished product is able to load almost all of the reports in under a second, even when pulled from an increasingly large dataset that uses over 800mb per month*.

Connecting It All

The call stats app is connected back into the Telecoms World PLC website via another API we created specifically for this project. This allows us to grant the website suitable permissions in order to handle the setup of new accounts and numbers on the fly from within the administration area. This time saving feature that benefits the client also ensures that customers are always receiving the most up-to-date information for their package. The app also has the ability to create multiple stats packages that can display any number of reports or overviews as required. This gives Telecoms World great flexibility in up-sells for their product range.

The call stats data powering this reporting is very important to Telecoms World PLC customers. Keeping this information safe has meant integrating replication master and slave databases together with an automated failover of the servers, ensuring that we are able to provide an extremely robust system.

TWPLC - Call Stats responsive dashboard

*Figures accurate at time of writing
Please Note: To protect client confidentiality the information containeed in the screenshots above are a representative example of the system only. None of the information represents real client data.