Kemp Support, how can we help?

The latest application delivery knowledge and expertise at your fingertips.

Progress DataDirect Hybrid Data Pipeline

1 Introduction

Progress DataDirect® Hybrid Data Pipeline provides simple, secure, and scalable access to universal data connectivity.

The Progress LoadMaster delivers an exceptional, cost effective, and easy to use solution which by employing intelligent server health checking, load balancing, high availability, and security can support an always-on application experience for DataDirect Hybrid Data Pipeline.

1.1 Document Purpose

This document provides the recommended LoadMaster settings used when providing load balancing for DataDirect Hybrid Data Pipeline. The Progress Support team is available to provide solutions for scenarios not explicitly defined. The Kemp Support site can be found at: https://support.kemptechnologies.com.

1.2 Intended Audience

This document is intended to be read by anyone who is interested in configuring the LoadMaster to optimize DataDirect Hybrid Data Pipeline.

2 Template

Kemp has developed a template containing our recommended settings for this workload. You can install this template to help create Virtual Services (VSs) because it automatically populates the settings. You can use the template to easily create the required VSs with the recommended settings. For some workloads, additional manual steps may be required such as assigning a certificate or applying port following. These steps are covered in the document, if needed.

You can remove templates after use and this will not affect deployed services. If needed, you can make changes to any of the VS settings after using the template.

Download released templates from the following page: LoadMaster Templates.

For more information and steps on how to import and use templates, refer to the Virtual Services and Templates, Feature Description on the Kemp Documentation page.

3 Architecture

Progress DataDirect deployments consist of multiple servers running Hybrid Data Pipeline behind a load balancer.

Architecture.png

4 Configure the LoadMaster

Refer to the sections below for details on some recommended global settings.

4.1 Enable Subnet Originating Requests Globally

It is best practice to enable the Subnet Originating Requests option globally.

In a one-armed setup (where the Virtual Service and Real Servers are on the same network/subnet) Subnet Originating Requests is usually not needed. However, enabling Subnet Originating Requests should not affect the routing in a one-armed setup.

In a two-armed setup where the Virtual Service is on network/subnet A, for example, and the Real Servers are on network B, Subnet Originating Requests should be enabled on LoadMasters with firmware version 7.1-16 and above.

Enable Subnet Originating Requests Globally v.2.png

When Subnet Originating Requests is enabled, the Real Server sees traffic originating from 10.20.20.21 (LoadMaster eth1 address) and responds correctly in most scenarios.

With Subnet Originating Requests disabled, the Real Server sees traffic originating from 10.0.0.15 (LoadMaster Virtual Service address on eth0) and responds to eth0 which could cause asymmetric routing.

When Subnet Originating Requests is enabled globally, it is automatically enabled on all Virtual Services. If the Subnet Originating Requests option is disabled globally, you can choose whether to enable Subnet Originating Requests on a per-Virtual Service basis.

To enable Subnet Originating Requests globally, follow the steps below:

1. In the main menu of the LoadMaster User Interface (UI), go to System Configuration > Miscellaneous Options > Network Options.

2. Select the Subnet Originating Requests check box.

5 Virtual Services

This step-by-step setup of Virtual Services (VSs) leverages the Progress Kemp application template for DataDirect Hybrid Data Pipeline. This template configures the Virtual Services to publish DataDirect with HTTP, HTTPS with TLS/SSL Offloading or HTTPS with TLS/SSL Reecryption.

The table in each section outlines the settings configured by the application template. You can use this information to manually configure Virtual Services or use the Progress Kemp LoadMaster Application Programming Interface (API) and automation tools.

5.1 Create the DataDirect Hybrid Data Pipeline – HTTPS Offloaded Virtual Service

The following are the steps involved and the recommended settings to configure the DataDirect Hybrid Data Pipeline HTTPS Offloaded Virtual Service.

  1. In the main menu of the LoadMaster User Interface (UI), go to Virtual Services > Add New.

    Virtual Services_01.png

  2. Type a valid Virtual Address.

  3. Select the DataDirect HDP HTTPS Offloaded template in the Use Template drop-down list.

  4. Click Add this Virtual Service.

5.1.1 Modify the DataDirect Hybrid Data Pipeline Default Sub Virtual Service

  1. Click Virtual Services and View/Modify Services.

  2. Click Modify on the DataDirect HDP HTTPS Offloaded Virtual Service on port 443.

  3. Expand the SSL Properties section.

  4. Select the certificate to use from Available Certificates and click the arrow (>) to move it to Assigned Certificates.

  5. Expand the SubVSs section.

  6. Click Modify for the HDP Offloaded Default Sub Virtual Service.

  7. Expand the Real Servers section.

  8. Click Add New.

  9. Type the Real Server Address.

  10. Enter 8080 for the Port.

  11. Click Add This Real Server.

  12. Repeat these steps to add more Real Servers as needed.

5.1.2 Modify the DataDirect Hybrid Data Pipeline Notification Sub Virtual Service

  1. Click Virtual Services and View/Modify Services.

  2. Click Modify on the DataDirect HDP HTTPS Offloaded Virtual Service on port 443.

  3. Expand the SubVSs section.

  4. Click Modify for the HDP Offloaded Notification Sub Virtual Service.

  5. Expand the Real Servers section.

  6. Click Add New.

  7. Type the Real Server Address.

  8. Enter 11280 for the Port.

  9. Click Add This Real Server.

  10. Repeat these steps to add more Real Servers as needed.

5.1.3 Modify the DataDirect Hybrid Data Pipeline OPA – Node 1 Sub Virtual Service

  1. Click Virtual Services and View/Modify Services.

  2. Click Modify on the DataDirect HDP HTTPS Offloaded Virtual Service on port 443.

  3. Expand the SubVSs section.

  4. Click Modify for the HDP Offloaded OPA Node 1 Sub Virtual Service.

  5. Expand the Real Servers section.

  6. Click Add New.

  7. Type the Real Server Address.

  8. Enter 40501 for the Port.

  9. Click Add This Real Server.

5.1.4 Modify the DataDirect Hybrid Data Pipeline OPA – Node 2 Sub Virtual Service

  1. Click Virtual Services and View/Modify Services.

  2. Click Modify on the DataDirect HDP HTTPS Offloaded Virtual Service on port 443.

  3. Expand the SubVSs section.

  4. Click Modify for the HDP Offloaded OPA Node 2 Sub Virtual Service.

  5. Expand the Real Servers section.

  6. Click Add New.

  7. Type the Real Server Address for the second DatatDirect Hybrid Data Pipeline Server.

  8. Enter 40501 for the Port.

  9. Click Add This Real Server.

  10. Repeat these steps to add more Real Servers as needed.

Additional OPA Sub Virtual Services can be created if more than two servers are in the environment.

5.1.5 Modify the DataDirect Hybrid Data Pipeline OPA Content Rules

Content Rules are created to direct traffic based on the path of the URL. The OPA Content Rules have unique paths for each environment and must be modified.

1. Click Rules and Checking and Content Rules.

2. Click Modify on the opa_Node1_xxxxx Content Rule.

3. Change the ADD_NODE_SERVER_IP to the unique path for the OPA Node 1 server.

4. Click Modify Rule to apply changes.

5. Repeat steps for remaining OPA Content Rules.

5.1.6 DataDirect Hybrid Data Pipeline HTTPS Offloaded Virtual Service Recommended API Settings (optional)

This table outlines the API parameters and values set using the Progress Kemp application template. You can use these settings with scripts and automation tools.

API Parameter

API Value

Root Virtual Service

port

443

prot

tcp
VStype http
Schedule lc
Persist none
SubnetOriginating 1
Forcel7 1
SSLAcceleration 1
TLSType 3
CipherSet BestPractices
HDP Offloaded Default - Sub Virtual Service
SubnetOriginating 1
Schedule lc
Persist src
PersistTimeOut 300
CheckType http
CheckUseGet 0
CheckPort 8080
CheckUrl /api/healthcheck
HDP Offloaded Notification - Sub Virtual Service
SubnetOriginating 1
Schedule lc
Persist src
PersistTimeOut 300
CheckType http
CheckUseGet 0
CheckPort 11280
HDP Offloaded OPA Node 1 - Sub Virtual Service
SubnetOriginating 1
Schedule lc
CheckType http
CheckUseGet 0
CheckPort 40501
HDP Offloaded OPA Node 2 - Sub Virtual Service
SubnetOriginating 1
Schedule lc
CheckType http
CheckUseGet 0
CheckPort 40501

5.2 Create the DataDirect Hybrid Data Pipeline – HTTPS Reencrypt Virtual Service

The following are the steps involved and the recommended settings to configure the DataDirect Hybrid Data Pipeline HTTPS Reencrypt Virtual Service.

  1. In the main menu of the LoadMaster User Interface (UI), go to Virtual Services > Add New.

    Virtual Services_02.png

  2. Type a valid Virtual Address.

  3. Select the DataDirect HDP HTTPS Reencrypt template in the Use Template drop-down list.

  4. Click Add this Virtual Service.

5.2.1 Modify the DataDirect Hybrid Data Pipeline Default Sub Virtual Service

  1. Click Virtual Services and View/Modify Services.

  2. Click Modify on the DataDirect HDP HTTPS Reencrypt Virtual Service on port 443.

  3. Expand the SSL Properties section.

  4. Select the certificate to use from Available Certificates and click the arrow (>) to move it to Assigned Certificates.

  5. Expand the SubVSs section.

  6. Click Modify for the HDP Reencrypt Default Sub Virtual Service.

  7. Expand the Real Servers section.

  8. Click Add New.

  9. Type the Real Server Address.

  10. Enter 8443 for the Port.

  11. Click Add This Real Server.

  12. Repeat these steps to add more Real Servers as needed.

5.2.2 Modify the DataDirect Hybrid Data Pipeline Notification Sub Virtual Service

  1. Click Virtual Services and View/Modify Services.

  2. Click Modify on the DataDirect HDP HTTPS Reencrypt Virtual Service on port 443.

  3. Expand the SubVSs section.

  4. Click Modify for the HDP Reencrypt Notification Sub Virtual Service.

  5. Expand the Real Servers section.

  6. Click Add New.

  7. Type the Real Server Address.

  8. Enter 11443 for the Port.

  9. Click Add This Real Server.

  10. Repeat these steps to add more Real Servers as needed.

5.2.3 Modify the DataDirect Hybrid Data Pipeline OPA – Node 1 Sub Virtual Service

  1. Click Virtual Services and View/Modify Services.

  2. Click Modify on the DataDirect HDP HTTPS Reencrypt Virtual Service on port 443.

  3. Expand the SubVSs  section.

  4. Click Modify for the HDP Reencrypt OPA Node 1 Sub Virtual Service.

  5. Expand the Real Servers section.

  6. Click Add New.

  7. Type the Real Server Address for the first DataDirect Hybrid Data Pipeline Server.

  8. Enter 40501 for the Port.

  9. Click Add This Real Server.

5.2.4 Modify the DataDirect Hybrid Data Pipeline OPA – Node 2 Sub Virtual Service

  1. Click Virtual Services and View/Modify Services.

  2. Click Modify on the DataDirect HDP HTTPS Reencrypt Virtual Service on port 443.

  3. Expand the SubVSs section.

  4. Click Modify for the HDP Reencrypt OPA Node 2 Sub Virtual Service.

  5. Expand the Real Servers section.

  6. Click Add New.

  7. Type the Real Server Address for the second DataDirect Hybrid Data Pipeline Server.

  8. Enter 40501 for the Port.

  9. Click Add This Real Server.

  10. Repeat these steps to add more Real Servers as needed.

Additional OPA Sub Virtual Services can be created if more than two servers are in the environment.

5.2.5 Modify the DataDirect Hybrid Data Pipeline OPA Content Rules

Content Rules are created to direct traffic based on the path of the URL. The OPA Content Rules have unique paths for each environment and must be modified.

  1. Click Rules and Checking and Content Rules.

  2. Click Modify on the opa_node1_xxxxx Content Rule.

  3. Change the ADD_NODE_SERVER_IP to the unique path for the OPA Node 1 server.

  4. Click Modify Rule to apply changes.

  5. Repeat steps for remaining OPA Content Rules.

5.2.6 DataDirect Hybrid Data Pipeline HTTPS Reencrypt Virtual Service Recommended API Settings (optional)

This table outlines the API parameters and values set using the Progress Kemp application template. You can use these settings with scripts and automation tools.

API Parameter

API Value

Root Virtual Service

port

443

prot

tcp
VStype http
Schedule lc
Persist none
SubnetOriginating 1
Forcel7 1
SSLAcceleration 1
SSLReencrypt 1
TLSType 3
CipherSet BestPractices
HDP Reencrypt Default - Sub Virtual Service
SubnetOriginating 1
Schedule lc
Persist src
PersistTimeOut 300
CheckType https
CheckUseGet 0
CheckPort 8443
CheckUrl /api/healthcheck
HDP Reencrypt Notification - Sub Virtual Service
SubnetOriginating 1
Schedule lc
Persist src
PersistTimeOut 300
CheckType https
CheckUseGet 0
CheckPort 11443
HDP Reencrypt OPA Node 1 - Sub Virtual Service
SubnetOriginating 1
Schedule lc
CheckType https
CheckUseGet 0
CheckPort 40501
HDP Reencrypt OPA Node 2 - Sub Virtual Service
SubnetOriginating 1
Schedule lc
CheckType https
CheckUseGet 0
CheckPort 40501

Last Updated Date

This document was last updated on 19 September 2022.


Comments