Connecting the ONE Platform to Cloud Pak for Data

Introduction

The ONE platform provides multivendor real-time data feeds in a single platform via a single API based on the publish/subscribe paradigm, hosted in the cloud or on-prem. Users can subscribe to different sources (e.g. Refinitiv, Bloomberg, or others) and, if they desire, publish their own data into the platform. Entitlement is provided via MECS, the Multivendor Entitlement Control System to ensure data integrity, confidentiality, and security.

 

Our goal with this Deployment Documentation is to provide you with instructions on how to set up the ONE Platform to ingest real-time streaming market data into Cloud Pak for Data via our MongoDB Adapter called One2MDB or by leveraging our native ONE API demo program to build a custom adapter.  Once this data is available in Cloud Pak for Data, you can use it with IBM Streams, Cognos Analytics, Watson Services (such as Watson Studio), and other IBM products to perform advanced analytics, machine learning, correlation, risk & liquidity management, algorithmic modelling, computations, and much more.  You can also display this data on a terminal, an HTML-5 application or in a spreadsheet.

 

Please reach out to BCC for a copy of One2MDB, the MongoDB adapter and/or the ONE API demo program package (contacts listed at the bottom).

 

 

 

Deployment instructions for One2MDB

The One2MDB adapter allows for data subscribed by the ONE Platform to be streamed into a MongoDB instance, e.g. a database running in Cloud Pak for Data. If the database is running in Cloud Pak for Data, the data can then be used in other services running in Cloud Pak for Data, e.g. IBM Streams or Cognos.

One2MDB can be configured using the config file provided within the download package. A file which contains a list of the instruments to be subscribed to needs to be provided by the user. The application is delivered as a compiled executable. Please provide information on your operating system when requesting the adapter package.

 

An instrument list example:

  • TREP|EUR=
  • TREP|GBP=
  • ZFP|EUR Curncy
  • ZFP|VOD LN Equity

(TREP = Refinitiv; ZFP = zero footprint (Bloomberg))

 

Getting started

Ingesting ONE Platform data into Cloud Pak for Data

 

Preparation:

Have the following information ready:

  1. ONE Platform
    • Username (needs MECS entry)
    • Password
    • Connection parameters
    • Application product key (needs MECS entry)
    • User entitlement for One2MDB usage
    • User entitlement for instruments requested in the list
    • Instrument subscription list
  1. MongoDB (in Cloud Pak Data)
    • IP address
    • Port
    • Database name
    • Collection name
  1. One2MDB
    • Log file location

 

Then, insert the information into the config file. Lines starting with a hash (#) are treated as comments.

 

Installation:

Unzip the distribution package and place the package folder to your desired location. After the config data is set to the user’s preferences and the instrument list is provided, the program can be started. It will subscribe to the instruments via the ONE platform and stream the incoming data into the configured MongoDB instance.

The log output should look something like this:

 

One2MDB Startup

Fig. 1: Sample log output of One2MDB startup

 

 

 

One2MDB sequential process

The application sends a login request to the ONE platform and connects to the configured MongoDB instance. All instruments in the given list are subscribed automatically; configurations are logged to the log file. As soon as data arrives, the messages are decoded and written into the database as key-value pairs and metadata. An example of a message (EUR to USD, TREP) to be written into the database:

 

 

Destination Topic '#P2P/v:config/THINKPAD-X1/14344/#00000001/U2E9_kpjnG/#/#'
SenderId CentOsFuji/HmsAdmin/HMS-T/BridgeDPU/#0#7636/2263500285
ApplicationMsgType CDR
ApplicationMessageId 213
SequenceNumber 3013
CorrelationId EUR=▲TREP
SenderTimestamp 1605267069531 (Fri Nov 13 2020 12:31:09)
Binary Attachment Map  

 
Key '#S#' (INT32) 1
Key '#M#' (STRING) StatusCode: None StatusText: [0002194818]All is well
Key '#Q#' (UINT64) 0
Key '#T#' (UINT64) 1605267065
Key '1' (UINT32) 213
Key '1000' (STRING) SPOT
Key '1001' (STRING) EURUSD

 

 

When closing the application all connections are disconnected cleanly.

 

In order to see the data in the database, open up your favourite MongoDB client. You should see something like this (example: subscribed to "TREP/CAD=", Canadian Dollar):

TREP Data in MongoDB

Fig. 2: data from subscription TREP/CAD= in MongoDB

Data Explanation

Fig. 3: Data explanation

 

 

 

 

Deployment instructions for the ONE API demo package

Once you receive the demo package, unpack it and inspect the README.

 

After unpacking the distribution archive file, the distribution top-level directory contains

four demo projects for the HmsCApi (the ONE API, written in C):

 

  1. HmsCApiDemo (demo for simple subscription)
  2. HmsCApiDemoDD (demo for retrieving data dictionary)
  3. HmsCApiMonDemo (demo for monitoring messages)
  4. HmsCApiPubSubDemo (demo for publishing sample data (and receiving them again by subscription))

 

The subdirectory Incl contains some include files for common use in the demo programs.

 

The demo will output the requested data to the console. In order to use the data, modify the data callback defined in DemoCallbacks.h: MyDataCB(HmsDataInfo_t * data) {…} or define your own callback and pass it to initCEvents(HmsCEvents_t * events) {…}, also defined in DemoCallbacks.h. Then, you can e.g. pass the data to a GUI or another API.

 

Compiling under Windows:

Change into the top-level directory (HmsCApiDemo) and open HmsCApDemo.sln under Visual Studio.

 

Compiling under Linux:

Change into the top-level directory (HmsCApiDemo), type 'make help' and follow the instructions.

 

The file “HmsConfig.h” will already be filled with sample values. Just run HmsCApiDemo or HmsCApiDemoDD to see some data. You can change these values according to the entitlement configuration assigned to your application.

Also, note that on Windows the Visual Studio solution might have to be retargeted to work: right-click on the solution within your IDE and select "Retarget solution".

 

The programs support command-line options to override values from HmsConfig.h and set some parameters.

Use option -h to show the list of supported options. Shell- / Batch- scripts run*.sh / run*.bat have been added to run the programs in a terminal, setting PATH on Windows properly and passing on command line arguments. On Linux, the programs are linked with a proper runtime library search path, so that setting LD_LIBRARY_PATH is not required (as long as the shared libraries are not moved to another place).

 

64/32 bit builds:

If the HmsCApi package in Components/HmsCApi/ is shipped with both 64 and 32 bit binaries, the demo programs can be build for both bit architectures. On Windows, the Visual Studio solution contains configurations for both cases. On Linux, set the variable BITS in the Makefile to 32 or override it on the command line, e.g.:

make BITS=32 relAll

 

 

Contact:

Rittik Wystup (rittik.wystup@bccgi.com)

Dani Müri (dani.muri@bccgi.com)

Bill Bierds (bill.bierds@bccgi.com)