How to set affinity between default unifier and its downstream operator

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

How to set affinity between default unifier and its downstream operator

Vivek Bhide
Hi

My application is connecting to Kafka topic with 2 partitions and there are corresponding KafkaSinglePortInputOperator instances. Since there are more than 1 instances, as per apex functionality, application has a default unifier as a next downstream operator when deployed. Problem is unifier operator and next downstream operator, dedupoperator, (defined in populateDag as downstream operator to KafkaSinglePortInputOperator) are getting deployed in same container. How can I make sure that unifier and dedupoperator are getting their individual instances? I know that, ANTI AFFINITY rule can be set in populateDag but how to set that rule with an operator which is present only at the run time (unifier). Is there any provision to prevent this from happening?

Regards
Vivek
Reply | Threaded
Open this post in threaded view
|

Re: How to set affinity between default unifier and its downstream operator

Sandesh Hegde
Why do you want unifier in the separate container? 

To put the unifier in the separate container, set UNIFIER_SINGLE_FINAL to true on the output port.

On Thu, Jun 29, 2017 at 10:56 PM Vivek Bhide <[hidden email]> wrote:
Hi

My application is connecting to Kafka topic with 2 partitions and there are
corresponding KafkaSinglePortInputOperator instances. Since there are more
than 1 instances, as per apex functionality, application has a default
unifier as a next downstream operator when deployed. Problem is unifier
operator and next downstream operator, dedupoperator, (defined in
populateDag as downstream operator to KafkaSinglePortInputOperator) are
getting deployed in same container. How can I make sure that unifier and
dedupoperator are getting their individual instances? I know that, ANTI
AFFINITY rule can be set in populateDag but how to set that rule with an
operator which is present only at the run time (unifier). Is there any
provision to prevent this from happening?

Regards
Vivek



--
View this message in context: http://apache-apex-users-list.78494.x6.nabble.com/How-to-set-affinity-between-default-unifier-and-its-downstream-operator-tp1771.html
Sent from the Apache Apex Users list mailing list archive at Nabble.com.