Kafka Producer

Description

Create an API to send a message to Kafka.

This Service uses the Kafka producer library to connect to an active instance of Kafka and send messages.

Note: Only a single message can be sent per rest call at this time

UI

Access the DataTrucker URL via a browser

Create a credentials

1. Go to Credentials in the sidebar > Kafka Credentials
2. Scroll down to Credential Management
3. Fill the form to create the credential
   1. Credential name – an arbitrary value to identify the credential object
   2. The hostname is a list of comma-separated brokers
      • example: ” ‘localhost:9092′,’host1:9091’ “
   3. Click on Create

Make an API

1. Go to IoT API in the Sidebar > Kafka APIs
2. Fill the form to create the API
   1. Resource Name: an arbitrary name to identify the resource
   2. The Credential linked to the resources
   3. Method: The type of REST Call
   4. Script: identifying the topic to send messages to.
   5. Validation Regex of Input values, input sanitization before querying
      1. Examples provided in the UI when you try to create a new API

Query the resource you created

################################ send an message to Kafka#############################URL: /api/v1/jobs/<resource name>TYPE: <method defined>HEADER: Authorization: "Bearer <JWT Token>"BODY (JSON): {        "key": "abcd",        "value": "123456"}
Response: 200 OK{   jsondata....}

As a CRD in Openshift / Kubernetes

For credentials use the the API below to the management end point

---apiVersion: datatrucker.datatrucker.io/v1kind: DatatruckerFlowmetadata:  name: datatruckerflow-samplespec:  Resources:    requests:      memory: "256Mi"      cpu: "250m"    limits:      memory: "256Mi"      cpu: "500m"  JobDefinitions:  - resourcename: kafka1    name: kafka1    type: IOT-Kafka-Producer    tenant: Admin    restmethod: PATCH    target_type:      topic: test    credentialname: kafkaconnect1    validations:      type: object      properties:        message:          type: string          pattern: "^[a-z0-9A-Z_]*$"          maxLength: 8  Keys:    configmap: placeholder  Scripts:    configmap: placeholder  Type: Job  DatatruckerConfig: datatruckerconfig-sample  Replicas: 1  API:    name: api    Image:      repository: docker.io      imageName: datatruckerio/datatrucker-api      tagName: latest
  

API

Create a credential via REST

URL: /api/v1/credentials/TYPE: POSTHEADER: Authorization: "Bearer <JWT Token>"BODY (JSON): {        "credentialname": "kafkaconnect1",        "type": "IOT-Kafka-Producer",        "hostname": "localhost:9092"}
Response: 201 OK

Create the API via REST

############################### create a Kafka Producer request #############################
URL: /api/v1/resourcesTYPE: POSTHEADER: Authorization: "Bearer <JWT Token>"BODY (JSON): {        "resourcename": "kafka1",        "type": "IOT-Kafka-Producer",        "restmethod": "PATCH",        "target_type": {          "topic": "test"        },        "credentialname": "kafkaconnect1",        "validations": {          "type": "object",          "properties": {            "message": {              "type": "string",              "pattern": "^[a-z0-9A-Z_]*$",              "maxLength": 8            }          }        }      }
Response: 201 OK