AWS上的独立Spectrum Symphony集群连接本地SMC的最佳实践

Starting from Spectrum Symphony 7.2.1, Symphony Multiple Cluster (SMC) feature allows outbound connection to the lone cluster. Here “outbound” is corresponding to historical “inbound”, meaning previously it was only allowed the lone cluster to initiate connection to the SMC master. Now SMC can also initiate connection to lone cluster, with some simple configurations only. You can get more details about “outbound connection” from Multicluster connections.

As more and more users are building clusters on Amazon AWS cloud, this article illustrates the best practice to setup connection from lone cluster on AWS with on-premises SMC.

• Outbound connection​

​

Configurations on the lone cluster master

-1. Configure ports in $EGO_CONFDIR/../../eservice/esc/conf/services/smcp.xml, including SMC_KD_PORT and SMC_PROXY_IDL_PORT. This port number of SMC_KD_PORT must match the port number of EGO_KD_PORT configured in the ego.conf file on the SMC master.

-2. Also specify SMC_PROXY_INBOUND_CONNECTION to Y in $EGO_CONFDIR/../../eservice/esc/conf/services/smcp.xml, meaning SMC master will initiate the connection to the lone cluster.

-3. In $EGO_CONFDIR/ego.conf, specify a port for EGO_KD_PORT.

-4. Update the public IP of SMC master into /etc/hosts. If your SMC master has one public IP and one private IP, in order to build up network connection, make sure the public IP is always before the private IP in /etc/hosts on the SMC master. Otherwise, SMCP will try to connect to the private IP and cause failure.

Configurations on the SMC master

-5. In $EGO_CONFDIR/ego.conf, specify a port for EGO_KD_PORT.

-6. In $EGO_CONFDIR/../../eservice/esc/conf/services/smcm.xml, configure a port for SMC_MASTER_IDL_PORT.

-7. Update the public IP of silo cluster master into /etc/hosts, like below. Since a host on AWS has an internal hostname and an external hostname, make sure both are mapping to the public IP.

Lone cluster firewall configuration on AWS

​-8. On AWS, go to Network & Security àSecurity Group, click on the target security group, go to Inbound policy. Add bypass ports defined in step 1 and 3. E.g., add TCP port 7870 from 0.0.0.0/0, meaning all external packets coming to 7870 will be bypassed.

SMC master firewall configuration

-9. Ensure that on SMC master the ports defined in step 5 and 6 are open. You can refer to Spectrum Symphony online document for more details: IBM Docs

-10. Restart SMCP service and the cluster on the lone cluster master, and restart SMCM service on the SMC.

-11. Add and join the lone cluster into the SMC on the SMC master, like below.

[root@VM_0_16_centos smc]# smcadmin cluster add -c cluster1 -m ip-172-31-17-22.ap-northeast-2.compute.internal -p 7870



-c cluster_name          Specifies the cluster name of the lone cluster
-m master_list           Specifies the fully qualified domain name of the master host of the cluster that is enabled with the multicluster proxy service. If you also have master candidate hosts, list them, separating each master with a comma.
-p vemkd_port            Specifies the VEMKD listening port for the cluster enabled with the multicluster proxy service

-12. Verify that the SMC connection type for the lone cluster is outbound:

 

After this verification, you may want to submit symping to verify workloads can be placed to this lone cluster. Follow steps below on the lone cluster to submit symping and replace the master_list with your defined IP and port.

[ec2-user@ip-172-31-17-22 logs]$ export SMC_GLOBAL_PLACEMENT=enabled

[ec2-user@ip-172-31-17-22 logs]$ export SMC_MASTER_CLUSTER_URL=master_list://*.*.*.*:xxxx

[ec2-user@ip-172-31-17-22 logs]$ symping -m 1

------------------------------------------------------------

IBM Spectrum Symphony Ping Utility.

 

Use this utility to test and verify that components are working and responsive.

Vary the workload to try out different IBM Spectrum Symphony workload conditions.

This client/service was developed using IBM Spectrum Symphony Developer Edition APIs.

 

Configuration         Value

 

Input message size      64 bytes
Output message size      128 bytes
Common data size       0 bytes
Client type          async
Number of test runs      1
Number of tasks per run    1
Task processing time     50 miliseconds
Consume CPU cycles      No
Session type         RecoverableAllHistoricalData
Data Compression       Disabled
Direct data transfer     Disabled

 

------------------------------------------------------------

Client connecting to IBM Spectrum Symphony...
Connected to Session Director.
Connection information:
User         : Admin
Application      : symping7.2.1

 

------------------------------------------------------------

Connected to Session Manager.
Creating 1 session...
Session created. ID: 12@cluster1    <=======session 12 was dispatched to cluster1

 

------------------------------------------------------------

Test Run 1.
Test for: Complete Workload Cycle
Sending 1 tasks and retrieving replies...

 

ExecHostName SI-PID TaskID       Execution(ms) TaskRoundtrip(ms)
ip-172-31-17 19882 1@12@cluster1    50.285    1,976.944

Number of tasks completed: 1
Connection to compute hosts OK.
SIs and SIMs OK.
Test Run 1 done.
Test Roundtrip: 2.87 seconds

 

------------------------------------------------------------

Workload Summary:
First task input sent  : 03/19/2019, 09:11:52.520
Last task output received: 03/19/2019, 09:11:54.643

 

Total Task Roundtrip:     2.026 seconds [Last task output received - First task input sent]

 

Number of compute hosts used: 1
Number of service instances used: 1

 

------------------------------------------------------------

Task Roundtrip Summary:
Shortest    1.977   ms (task 1@12@cluster1 on host ip-172-31-17-22.ap-northeast-2.c)
Longest    1.977   ms (task 1@12@cluster1 on host ip-172-31-17-22.ap-northeast-2.c)
Average    1.977   ms

 

------------------------------------------------------------

Host Summary:
Compute Host          SI-PID  Number of Tasks
ip-172-31-17-22.ap-northeast-2.c 19882   1

 

------------------------------------------------------------

Configuration Summary:
Application: symping7.2.1   Session ID: 12@cluster1

• Inbound connection

All configurations are the same to outbound above, except that you need to specify SMC_PROXY_INBOUND_CONNECTION to N in $EGO_CONFDIR/../../eservice/esc/conf/services/smcp.xml, meaning lone cluster master will initiate the connection to the SMC master.