6.2.2. Bloomberg

A Bloomberg Kafka Connector to subscribe to Bloomberg feeds and source via the Bloomberg labs open API data and write to Kafka.

6.2.2.1. Prerequisites

  • Apache Kafka
  • Kafka Connect
  • Bloomberg subscription

6.2.2.2. Connector QuickStart

The easiest way to quick-start with this connector is to try out the Lenses Development Environment as it’s a docker image ready for your laptop, runs Kafka and Kafka Connect, and has the connector already available in the classpath

Select Connectors -> New Connector -> Bloomberg and paste the following configuration

Note

You will need a Bloomberg subscription to proceed

name=bloomberg-source
tasks.max=1
connector.class=com.datamountaineer.streamreactor.connect.bloomberg.BloombergSourceConnector
connect.bloomberg.server.host=localhost
connect.bloomberg.server.port=8194
connect.bloomberg.service.uri=//blp/mkdata
connect.bloomberg.subscriptions=AAPL US Equity:LAST_PRICE,BID,ASK;IBM US Equity:BID,ASK,HIGH,LOW,OPEN
connect.bloomberg.authentication.mode=USER_AND_APPLICATION
connect.bloomberg.authentication.mode=bloomberg
connect.bloomberg.buffer.size=4096
connect.bloomberg.payload.type=avro
connect.bloomberg.kafka.topic=bloomberg

The bloomberg-source.properties file defines:

  1. The connector name.
  2. The class containing the connector.
  3. The number of tasks the connector is allowed to start.
  4. The Bloomberg server host.
  5. The Bloomberg server port.
  6. The Bloomberg service uri.
  7. The subscription keys to subscribe to.
  8. The topic to write to.
  9. The buffer size for the Bloomberg API to buffer events in.
  10. The authentication mode.

6.2.2.3. Features

The Source Connector allows subscriptions to BPipe mkdata and refdata endpoints to feed data into Kafka.

6.2.2.4. Configurations

Config Description Type
connect.bloomberg.server.host The bloomberg endpoint host to connect to string
connect.bloomberg.server.port The bloomberg endpoint port to connect to string
connect.bloomberg.service.uri Which Bloomberg service to connect to. Can be //blp/mkdata or //blp/refdata string
connect.bloomberg.authentication.mode
The mode to authentication against the Bloomberg server.
Either APPLICATION_ONLY or USER_AND_APPLICATION
string
connect.bloomberg.subscriptions
Specifies which ticker subscription to make. The format is TICKER:FIELD,FIELD,..;
e.g. AAPL US Equity:LAST_PRICE;IBM US Equity:BID
string
connect.bloomberg.kafka.topic The The topic to write Bloomberg data to string
connect.bloomberg.payload.type
Specifies the way the information is serialized and sent over kafka
There are two modes supported: json(default) and avro
string

Optional configurations

Config Description Type Default
connect.bloomberg.buffer.size
The buffer accumulating the data updates received from Bloomberg. If not provided
it will default to 2048. If the buffer is full and a new update will be received it
won’t be added to the buffer until it is first drained
int 2048
connect.progress.enabled Enables the output for how many records have been processed boolean false

6.2.2.5. Schema Evolution

TODO

6.2.2.6. TroubleShooting

Please review the FAQs and join our slack channel.