Consumer Policy Document

Version No. 1.0

 Consumer Policy Configuration 1

Confidential and Proprietary

Copyright © 2023, Acidaes Solutions Pvt. Ltd. All Rights Reserved.

The management team of Acidaes Solutions Pvt. Limited has prepared this business document and it
is being furnished to selected individuals within the customer organization for the sole purpose of
proposal evaluation for the sale of BUSINESSNEXT.

This document is confidential and contains ideas, concepts, processes, and other information that
Acidaes Solutions considers proprietary. Readers are to treat the information contained herein as
confidential and may not disseminate; copy or reproduce it in any form without the expressed
written permission of Acidaes Solutions Pvt. Limited. ‘Acidaes’, ‘BUSINESSNEXT’, ‘Simplifying
Technology’ and ‘business efficiency, on demand’ are applied trademarks of Acidaes Solutions Pvt.
Ltd. All other trademarks are the property of their respective owners.

www.businessnext.com
 Consumer Policy Configuration 2

Revision History

DOCUMENT INFORMATION
DOCUMENT TITLE Consumer Policy Configuration

DEPARTMENT Product Development Group (PDG)

AUTHOR Content Team, PAG

DOCUMENT AMENDMENT RECORD

VERSION NO. PUBLISHED DATE VERIFIED BY REVISION TRACKING NOTES
1.0 February 2023 Sandeep K Tripathi Technical Document Drafted

www.businessnext.com
 Consumer Policy Configuration 3

Contents

Executive Summary................................................................................................................................ 4
About BUSINESSNEXT ............................................................................................................. 4

Introduction ........................................................................................................................................... 5
Dead Letter Queue (DLQ) ....................................................................................................... 5

Business Level Benefits ........................................................................................................... 5

Prerequisites ........................................................................................................................... 6

Consumer Policy Configuration from Backend ...................................................................................... 7

ConsumerPolicy ...................................................................................................................... 7
Sample Query ................................................................................................................. 8

ConsumerGroupDetail ............................................................................................................ 9
Sample Query ............................................................................................................... 10

ConsumerPolicyMembers ..................................................................................................... 11
Sample Query ............................................................................................................... 11

Error Code Table ................................................................................................................... 12

Setting up Retry and DL Queue Configuration for a Digital Journey ................................................... 13

Troubleshooting Guidelines .................................................................................................. 18

www.businessnext.com
 Consumer Policy Configuration 4

Executive Summary
About BUSINESSNEXT

BUSINESSNEXT has proven credentials in

empowering the BFSI segment with its cutting-
edge offerings to boost customer experience and
drive employee engagement. It has to its credit
among the largest Banking CRM implementations
in the world with 35+ countries. It’s a single
platform to drive digital transformations with a
variety of product lines including CRMNEXT,
CUSTOMERNEXT, ORIGINATIONNEXT, BOTNEXT,
DATANEXT, BRANCHNEXT, RISKNEXT, and so on.
With more than a million financial service
associates and a billion customers supported by
the platform globally, we effectively recalibrate
banks' potential to grow engagement, drive
profitability and deliver innovation.
Established in 2006, the company is PCI-DSS certified and named Challenger in Gartner’s 2021 Magic
Quadrant for Sales Force Automation (SFA). This is our fifth consecutive year of being assessed by
Gartner and a commendable shift from the Visionaries quadrant to the Challengers quadrant in 2021
“based on its scopes of advanced capabilities, such as bot technology, white–space analytics, and
machine learning. We have also badged the title of Niche Players in Gartner’s 2021 Magic Quadrant
for Customer Engagement, B2B Marketing Automation Platform, and CRM Lead Management.
Gartner calls out high usability, easy navigation, and customization possibilities as our strengths along
with our ability to improve customer experience.

www.businessnext.com
 Consumer Policy Configuration 5

Introduction
We cannot expect data to perfectly run every time, so whenever anything wrong happens, we wish
our pipelines to handle such situations gracefully. Kafka comes with error-handling options and has
the ability to redirect messages to a Dead Letter Queue (DLQ) in such adverse circumstances.

Dead Letter Queue (DLQ)

The Dead Letter Queue (DLQ) is an implementation of a service within a data streaming platform or
a messaging system to store messages that do not get processed successfully. So, instead of dumping
these messages passively, the system moves them to a Dead Letter Queue.

As per our DLQ management, DLQ has its own attempts to process a message in which it passes the
message again to the main Topic/Consumer for the processing. After specified attempts, if a message
gets failed at DLQ level, then it will be dumped to the database (ConsumerFailedLogs).

A pictorial representation of how DLQ works in CUSTOMERNEXT:

Business Level Benefits

This document helps our team to configure the Consumer Policy through backend without UI.

www.businessnext.com
 Consumer Policy Configuration 6

Prerequisites

You are required to execute manual script to create the consumer-policy related tables.

www.businessnext.com
 Consumer Policy Configuration 7

Consumer Policy Configuration from Backend

A representation of Relationship diagram of Consumer Policy Table.

Below is the description of how to configure Consumer Policy from backend:

ConsumerPolicy

S.No. Configuration Parameter Description

1. SubscriberId The owner ID of the client.
2. PolicyId It is a strategy that explains which next consumer will run
after a message gets failed by the main consumer or retry.

www.businessnext.com
 Consumer Policy Configuration 8

3. Name It is a Data Source Name created from the Kafka

connection or Data Source in BusinessNext.
4. MainConsumerGroupId This is a data source ID of Consumer which is configured
for the main topic. You can get the data source ID from
the MashupDataSource table
5. DLQConsumerGroupId This is a data source ID of Consumer which is configured
for the DLQ topic. You can get the data source ID from the
MashupDataSource table.
6. DLQBigDataConnectionId This is a Kafka connection ID. You can get the connection
ID from MashupConnection table.
7. Description
8. CreatedOn The current date and time when the data is inserted in
the table.
9. CreatedBy The name of the user who inserted the data in the table.
10. LastModifiedOn The date and time of the last modification are done in the
table.
11. LastModifiedBy The name of the user who last modified the data in the
table.

Sample Query

INSERT INTO [dbo].[ConsumerPolicy]

([SubscriberId],[PolicyId],[Name],[MainConsumerGroupId],[DLQConsumerGroupId],[DLQBigData
ConnectionId],[Description]
,[CreatedOn],[CreatedBy],[LastModifiedOn],[LastModifiedBy])
VALUES
(-1--Enter subscriberId of the company
,1 --Enter integer value which should be same for all topics/consumers
,'PolicyNameRelevant' --Enter any relevantName
,111111 --Enter DataSourceId
,11111 --Enter DLQ consumerDataSourceId
,11111 --Enter connectionId under which DLQ consumer is configured
,'Consumer' --EnterDescription
,getdate()
,1
,GETDATE()
,1
GO

www.businessnext.com
 Consumer Policy Configuration 9

As of now, these entries should be entered manually through backend.

ConsumerGroupDetail

S.No. Configuration Parameter Description

1. SubscriberId The owner ID of the company.
2. PolicyId It is a strategy that explains which next consumer will run
after a message gets failed by the main consumer or retry.
3. ConsumerGroupId This is a data source ID of Consumer which is configured
for the topic. You can get the data source ID from the
MashupDataSource table
4. Name It is a Data Source Name created from the Kafka
connection or Data Source in BusinessNext.
5. ConsumerType This is the type defined for Consumers as below:
Main Consumer - 1
Retry Consumer - 2
DLQ Consumer - 3
6. Topic It is the Topic name from where the Consumer will
consume messages.
7. MinConsumers The count of the minimum number of Consumer
instances.
8. MaxConsumers The count of the maximum number of Consumer
instances.
Note: For future use

www.businessnext.com
 Consumer Policy Configuration 10

9. MaxRetryAttempts The number of times the message is retried in the same

topic if failed.
Note: This is not applicable for Consumer Type - 1 (Main
Consumer)
10. ReadDelay The time interval between each retry attempt for the
failed message in a particular topic. However, this time is
in milliseconds.
11. RetryConnectionId This is a data source ID of Consumer which is configured
for the retry topic. You can get the data source ID from
the MashupDataSource table
12. RetryConsumerGroupId This is a data source ID of Consumer which is configured
for the retry topic. You can get the data source ID from
the MashupDataSource table
13. Sequence It is the sequence of the topics.
14. Description It is the description of the policy.
15. AdditionalSettings Note: For future purpose only
16. AddedOn The date and time when the data is added in the table.
17. AddedBy The name of the user who added the data in the table.
18. LastModifiedOn The date and time of the last modification are done in the
table.
19. LastModifiedBy The name of the user who last modified the data in the
table.

Sample Query

INSERT INTO [dbo].[ConsumerPolicyMembers]

([SubscriberId],[PolicyId],[ConsumerGroupId],[AddedOn],[AddedBy])
VALUES
(0 --Enter subscriberId of the company
,1 --Enter integer value which should be same for all topics/consumers
,-1 --Enter datasourceid for the consumer
,1
,getdate()
GO

As of now, these entries should be entered manually through backend.

www.businessnext.com
 Consumer Policy Configuration 11

ConsumerPolicyMembers

S.No. Configuration Parameter Description

1. SubscriberId The owner ID of the company.
2. PolicyId It is a strategy that explains which next consumer will run
after a message gets failed by the main consumer or retry.
3. ConsumerGroupId This is a data source ID of Consumer which is configured
for the topic. You can get the data source ID from the
MashupDataSource table
4. AddedOn The date and time when the data is added in the table.
5. AddedBy The name of the user who added the data in the table.

Sample Query
INSERT INTO [dbo].[ConsumerGroupDetail]
([SubscriberId],[PolicyId],[ConsumerGroupId],[Name],[ConsumerType],[Topic],[MinConsumers],
[MaxConsumers],[MaxRetryAttempt]
,[ReadDelay],[RetryConnectionId],[RetryConsumerGroupId],[Sequence],[Description],[Additiona
lSettings],[AddedOn],[AddedBy],[LastModifiedOn],[LastModifiedBy])
VALUES
(-1
,1 --Enter PolicyId
,-1 --Enter datasourceId of the consumer
,'DataSourceName'
,-1 -- Enter consumerType as per the type of consumer
,'TopicName'
,-1 --Enter mininumNumber of consumer
,-1 --Enter MaxNumberOfConsumer
,-1 --Enter Maximum retry attempt
,0 --Enter time difference between retry
,-1 --Enter connectionId
,-1 --Enter DataSurceId
,-1 --Enter Sequence
,'Description'
,'' --For Future use
,1
,getdate()
,1
GOwww.businessnext.com
 Consumer Policy Configuration 12

As of now, these entries should be entered manually through backend.

Error Code Table

Error Code Table is used to maintain the relationship between Error Code, and retryable and non-
retryable table.
If the value in the Type column is 1, this means that it is retryable, and if it is 2, it means it is non-
retryable
Type = 1 – Retryable
Type = 2 – Non Retryable

S.No. Configuration Parameter Description

ErrorSource Define whether the error is coming from CRM, Consumer
Service, etc.
ErrorCode An alpha numeric code of the error
ErrorMessage It is the description of error
Type 1 – Retryable
2 – Non Retryable
Category Note: To categorize the error, if it is a logical, validation
or configurational error. (Not in use. For future purpose
only)
AddedBy The name of the user who added the data in the table.
AddedOn The date and time when the data is added in the table.
UpdatedBy The name of the user who updated the data in the table.
UpdatedOn The date and time when the data is updated in the table.

As of now, these entries should be entered manually through backend.

www.businessnext.com
 Consumer Policy Configuration 13

Setting Configuration for Consumer Service

{
"LogLevel": {
"Default": "Information",
"Microsoft.Hosting.Lifetime": "Information"
},

"ConsumerProvider": "Kafka",
//"ConnectionType": "Sql",
//"DataRepository": "mongo",
"ProcessorType": "sdk", //restapi, sdk

"ConsumerLogsSettings": {
"IsLoggingEnabled": "true",
"KafkaConsumerLogger": "KafkaConsumerLogger" //GRPC service URL which writes for metrics
},

"OpenTelemetrySettings": {
"ServiceName": "ConsumerService",
"EnableTracing": "true",
"OtelEndpoint": "http://192.168.0.100:30000" //GRPC service URL which writes for metrics
},

"MetricsSettings": {
"EnableMonitoring": "false",
"otelMetricsEndPoint": "http://192.168.0.100:30000" //GRPC service URL which writes for metrics
},

//"ConnectionString": "data source=192.168.0.100;initial catalog=CUSTOMERNEXT;password=asde123;persist

security info=True;user id=sa;packet size=4096;enlist=false;",
"MetadataRepositorySettings": {
"MetadataRepositoryType": "SQL",
"Policies": "1, 2",
"ConnectionStrings": {
"ConnectionString": "data source=192.168.0.7000;initial catalog=crmnext;password=abc2334;persist security
info=True;user id=crmnext;packet size=4096;enlist=false;",
"providerName": "System.Data.SqlClient"
},
"MongoDataRepositorySettings": {
"DbName": "crmnext",
"UserName": "root",
"Password": "kZGXQvp4KE",
"AuthenticationDatabaseName": "admin",
"HostIP": "192.168.0.124",
"Port": "31425"
}

www.businessnext.com
 Consumer Policy Configuration 14

"HostIP": "192.168.0.100",
"Port": "30000"
}
},

"DataRepositorySettings": {
"DataRepositoryType": "sql",
"ConnectionStrings": {
"ConnectionString": "data source=192.168.0.700;initial catalog=hdfc_crmnext;password=abc12344;persist
security info=True;user id=crmnext;packet size=4096;enlist=false;",
"providerName": "System.Data.SqlClient"
},
"MongoDataRepositorySettings": {
"DbName": "crmnext",
"UserName": "root",
"Password": "kZGXQvp4KE",
"AuthenticationDatabaseName": "admin",
"HostIP": "192.168.0.100",
"Port": "30000”
}
},

//"ConnectionStrings": {
// "ConsumerDataSourceIds": "",
// "ConnectionString": "data source=192.168.0.280;initial catalog=crmnext;password=abc123;persist security
info=True;user id=crmnext;packet size=4096;enlist=false;",
// "providerName": "System.Data.SqlClient"
//},

//"MongoDataRepositorySettings": {
// "DbName": "crmnext",
// "UserName": "root",
// "Password": "kZGXQvp4KE",
// "AuthenticationDatabaseName": "admin",
// "HostIP": "192.168.0.100",
// "Port": "3000"
//},

"ApiConfiguration": {
"Keyword": "CRM",
"UserId": "hdfc2@crmnext.com",
"Password": "acid_qa",
"crmNextApiURL": "https://abc.crmnext.com/rest/",
"LoginMedium": "WebService"
}

www.businessnext.com
 Consumer Policy Configuration 15

S.No. JSON Parameters Description Example

1. LogLevel Default Information

Microsoft. Information
Hosting.Lifetime
2. ConsumerProvi As of now, we are using kafka
der 'Kafka' as
ConsumerProvider
3. ProcessorType To run consumer service SDK
On-Prem, the processor
type will be set as SDK.
4. OpenTelemetry This key is used to provide
Settings tracing and logging.

ServiceName This key is used to provide ConsumerService_QA

OTEL ServiceName. By
default, a consumer service
trace is created with
**Acidaes_Messaging_Con
sumer**.
EnableTracing This key is used for on and True
off-tracing/logging.
OtelEndpoint It is used to provide IP http://192.168.0.100:30000
where logs/traces will be
written.
5. MetricsSettings This key is used to provide
details to create
metrics/counters.
EnableMonitorin True/False false
g

otelMetricsEndP Enter otel end point here. http://192.168.0.100:30000

oint

6. MetadataReposi This key is used to define

torySettings the details about where
consumer configuration
will be picked up.
Metadata It can be either SQL
RepositoryType SQL/Oracle/Mongo. As of
now, we are using only
SQL.
Policies PolicyId to run consumer 1, 2
with as stated above.

www.businessnext.com
 Consumer Policy Configuration 16

ConnectionStrin ConnectionString If data

gs MetadataRepositoryType source=192.168.0.7000;initi
is SQL, then this connection al
string will be used. catalog=crmnext;password=
abc2334;persist security
info=True;user
id=crmnext;packet
size=4096;enlist=false;

providerName If System.Data.SqlClient
MetadataRepositoryType
is MongoDB, then this
connection string will be
used.
MongoDataRep This is the connection
ositorySettings where consumer
configuration is created.
DbName crmnext
Details of MongoDb will be
UserName filled in here. root

Password kZGXQvp4KE

Authentication admin
DatabaseName

HostIP 192.168.0.100

Port 30000

7. DataRepository It contains the settings of

Settings the database where
consumer logs will be
created to keep track of
consumers, retry
consumers, DQL
consumers, messages,
message sequences, and
more details.
DataRepositoryT It can be SQL or MongoDB. SQL
ype

ConnectionString If Datarepositorytype is data

s SQL, this key will be used source=192.168.0.700;initial
catalog=hdfc_crmnext;pass

www.businessnext.com
 Consumer Policy Configuration 17

word=abc12344;persist
security info=True;user
id=crmnext;packet
size=4096;enlist=false;

providerName system.data.sqlclient System.Data.SqlClient

MongoDataRep This is the connection

ositorySettings where consumer
configuration is created.
DbName crmnext

UserName root

Password kZGXQvp4KE
Details of MongoDB will be
AuthenticationD filled in here. admin
atabaseName

HostIP 192.168.0.100

Port 30000

8. ApiConfiguratio It contains relevant

n information to login into
restapi and create an
object If we have set
processortype as 'restapi'.
Keyword We have created this key CRM
to keep track of errorcodes
generated by restapi/sdk
means CRM (can be
changed).
UserId This is to provide restapi "hdfc2@crmnext.com
UserId to generate a token.
Password This is to provide restapi acid_qa
Password to generate a
token.
crmNextApiURL This is used to enter restapi https://abc.crmnext.com/re
URL. st/
LoginMedium As of now, this is set as WebService
'WebService'.

www.businessnext.com
 Consumer Policy Configuration 18

Configuration Settings for Consumer Service as SDK

Update the connection string in the file:

Acidaes.Messaging.Consumer.Service.exe.config
This connection string will be used for the lead creation.

Troubleshooting Guidelines

There are some troubleshooting guidelines while configuring Consumer Policy from backend as listed
below:
• Topic mentioned in the Main Consumer must be same with the Producer
• ConsumerType in ConsumerGroupDetail must be filled correct as per the details provided in
the table
• Type in ErrorCodes must be correct as mentioned in the table description
• ConsumerGroupId must be correct in all the table

www.businessnext.com