Apps can register data hooks that will be called by Rimdian when specific events occur.

As opposed to tasks that are executed in the background, data hooks are called in real-time in the data processing pipeline and are expected to return a response within a few seconds.

A common use case for data hooks is to enrich or transform data before it is inserted by Rimdian, or to forward data to a third-party system.

​
Data hooks types

Rimdian supports the following data hook types:

  • on_success - called when a data log item is successfully processed (created, updated…).
  • on_validation - called when a data log item about to be validated (before it is processed), and can be used to enrich/transform/reject the data item.

​
Hook on_success

It is important to return a response within 7 seconds. If the response is not received within a few seconds, the data hook will be retried 3 times before being dropped!

When a on_success data hook is called, Rimdian will send a POST request to your app webhook_endpoint with the following payload:

{
    "api_endpoint": "https://RIMDIAN_API_ENDPOINT",
    "collector_endpoint": "https://RIMDIAN_COLLECTOR_ENDPOINT",
    "workspace_id": "acme_workspace1",
    "app_id": "app_myapp",
    "app_state": {
        "key1": "value1",
        "key2": "value2"
    },
    "kind": "data_hook",
    "data_hook": {
        "data_hook_id": "app_myapp_onsuccess",
        "data_hook_name": "Forward data on success",
        "data_hook_on": "on_success",
        "data_log_id": "1234567890",
        "data_log_kind": "order",
        "data_log_action": "create",
        "data_log_item": "{ ... raw JSON item string ... }",
        "data_log_item_id": "xxxxxxxxx",
        "data_log_item_external_id": "order_123",
        "user": {
            "id": "xxxxxx",
            "external_id": "user_123",
            "is_authenticated": true,
            "created_at": "2024-01-01T00:00:00Z",
            "updated_at": "2024-01-01T00:00:00Z",
            "email": "john@website.com",
            "first_name": "John",
            "last_name": "Doe",
            "phone": "+1234567890",
            "country": "US",
            "language": "en",
            "timezone": "America/New_York",
            ... other user properties
        }
    }
}

The full user object will be included if the data log item is associated with a user.

The data_log_item is the JSON-escaped string value of the item sent to the Collector.

​
Expected response

{
    "is_done": true,
    "is_error": false,
    "message": "done"
}

Returning is_done as false will cause the data hook to be retried.

​
Hook on_validation

It is important to return a response within 5 seconds. If the response is not received within a few seconds, the data hook won’t be retried!

When a on_validation data hook is called, Rimdian will send a POST request to your app webhook_endpoint with the following payload:

{
    "api_endpoint": "https://RIMDIAN_API_ENDPOINT",
    "collector_endpoint": "https://RIMDIAN_COLLECTOR_ENDPOINT",
    "workspace_id": "acme_workspace1",
    "app_id": "app_myapp",
    "app_state": {
        "key1": "value1",
        "key2": "value2"
    },
    "kind": "data_hook",
    "data_hook": {
        "data_hook_id": "app_myapp_onvalidation",
        "data_hook_name": "Enrich data on validation",
        "data_hook_on": "on_validation",
        "data_log_id": "1234567890",
        "data_log_kind": "order",
        "data_log_item": "{ ... raw JSON item string ... }"
    }
}

The data_log_item is the JSON-escaped string value of the item sent to the Collector.

​
Expected response for rejecting the data

{
    "action": "reject",
    "message": "Optional message to explain why the data was rejected"
}

​
Expected response for enriching the data

{
    "action": "update",
    "message": "Optional message to explain why the data was updated",
    "updated_item": "{ ... raw updated JSON item string ... }"
}
The updated_item field should be a JSON string and not a JSON object.

​
Expected response for accepting the data

You just need to return a 200 OK status code with an empty body to accept the data.

TasksSQL queries