Market Data Platform
The Challenge
The Challenge
The client was using several market data providers and wanted to store the data into a unified format which is provided to different internal consumers as SQL tables. ROITI was approached to design and develop the data collection, transformation, and data feed to OpenLink Endur for price data.
The Approach
The Approach
To ensure time-to-market and cost efficiency we decided to use Talend as the central ETL tool for the project. The incoming data was sourced via multiple channels – FTP, HTTP and file share – with one metadata driven download job. The data was than transformed in a unique format and exposed to stage tables via multiple format specific import jobs. The data was than mapped to the Endur pricing curves, before generating ready-to-import csv. files, which Endur’s standard import functionality could handle. The monitoring of the whole process was integrated into Splunk.
The Impact
The Impact
With the centralized solution, outages and failed price imports have been minimized. The automation of data collection and the event-based data feed have decreased the throughput time and have allowed Endur operation on near-real time price data. The utilization of Talend has allowed more sources to be easily integrated into the process with minimal effort.
Optimized solutions for every case
Contact us and learn how can we help
