Consulenza, soluzioni e

servizi per l’ICT

©Copyright 2013 Lutech Spa

Matteo RedaelliAnkara 25-03-2015C2Sense Meeting

Task D 3.2

Kafka quick introduction

Demostration Structure

Producer Web Service

Consumer Web Service

Producer Client

Consumer Client

Web Console

Agenda

Task 3.2 Protocol Interoperability

This task’s objective is to achieve the protocol interoperability of emergency applications/systems and sensors

using heterogeneous protocols. In this respect, “Protocol interoperability” addresses the transport level protocols

such as TCP/IP, HTTP, SOAP, REST or SMTP and is in charge of end-to-end delivery of messages. C2-SENSE

will use Web service protocols for this layer by exposing the proprietary services of emergency applications and

organizations as “Web services”. In this task, first the detailed Operational and Core Web Service categories

will be identified under the supervision of SAGEM. In this layer only the abstract definition of these services

will be developed. Their interface schemas describing inputs/outputs in detail will be defined in Task 3.3 and

the services will be implemented in Task 3.4). After that for these services related protocol interoperability

profiles will be developed and based on these profiles corresponding Protocol Adapters will be implemented.

These protocol adapters will be used to access emergency applications/systems and sensors in the C2-SENSE

architecture through Web Service protocols. There will be two types of Protocol Adapters: (1) Sensor Protocol

Adapters (to be developed by AIT) and (2) Emergency Applications/Systems Protocol Adapters (to be developed

by SAGEM).

Furthermore, the communication among the components and services of the C2-SENSE Architecture will be based on an Enterprise Service Bus (ESB), which will be developed by LUTECH.

Task 3.2 Protocol Interoperability

Kafka Introduction

Kafka as ESB:– Fast

Could handle hundreds of messages from different sources.

– Scalable It has an architecture that could evolve and grow

without losing time.– Durable

The possibility of losing messages is reduced.– Distributed by Design

It has possibility of retrieving messages in a second moment.

Introduction – Connection with C2Sense

Kafka Architecture - Glossary (1)

Producer = who publish messagesConsumer = who read messagesBroker = nodeTopic = message categoriesPartition = ordered, immutable sequence of messages

Kafka Architecture(2)

• Queue, in which each message is sent only to one reader, even if there are many

• Publish/Subscribe, in which message is sent to any reader interested in it

Producer Web Service

Producer Web Service:– Connector for sensors– Web Service is connected to the ESB

Producer Web Service

Consumer Web Service

Consumer Web Service:– Connector for retrieving messages

from the ESB– Web Service is connected to the ESB

Consumer Web Service

Producer Client

Producer Client– Very Simple Python script– Sensor like client

Producer Client

tstamp topic from lat lng hydrolevel

StringTimestamp

String“Sensor1”

StringFrom Water Sensor

Stringlatitude

Stringlongitudine

IntRandom number

Consumer Client

Consumer Client– Very Simple Python script– Retrieving messages from Web Service

Consumer Client

maxNumOfMsgs fromTopic fromPartition

Int100

String“Sensor1”

Int 1

Web Console

Web Console– Topic– Brokers– SettingsCould be a consumer

Web Console

Dove siamo

Direzione e Sede OperativaVia W.A. Mozart 47,

20093 Cologno Monzese (MI)Tel. +39-02-25427011

Fax +39-02-25427090

Sede di RomaVia A. Mantegna 4,

Parco Leonardo00054 Fiumicino (RM)

Tel. +39-06-227501Fax +39-06-22771542

Altre sedi aTorino, Modena, Firenze, Bari

Email info@lutech.it Web http://www.lutech.it