Topic
  • 12 replies
  • Latest Post - ‏2013-04-04T17:25:10Z by SystemAdmin
SystemAdmin
SystemAdmin
9855 Posts

Pinned topic TDS 6.2 Peer Replication Issue/Question

‏2013-04-02T11:48:41Z |
Hello folk,
I have setup an TDS 6.2 (on z/Linux)peer to peer replication, which by the way works fine.
I am now trying to restrict the replication by excluding a specific subtree. Is there an elegant way to achieve this?
For illustration here is how it should looks like:

Peer1 and peer2 have a replication agreement on the subtree "o=sample,c=de".
Now I would like to exclude (prevent peer1 and peer2 to replicate this concrete subtree) the following subtree (and all the entries under it) from the replication: ou=myOrgUnit,o=sample,c=de.

Can someone help?

Cheers, Emile
Updated on 2013-04-04T17:25:10Z at 2013-04-04T17:25:10Z by SystemAdmin
  • yn2000
    yn2000
    1086 Posts

    Re: TDS 6.2 Peer Replication Issue/Question

    ‏2013-04-02T13:26:18Z  
    Unfortunately, you cannot.
    You can only set the other way around, which is the other sub-tree that you want to include in the replication process, even though you may have many of them.

    Rgds. YN.
  • SystemAdmin
    SystemAdmin
    9855 Posts

    Re: TDS 6.2 Peer Replication Issue/Question

    ‏2013-04-02T14:35:57Z  
    • yn2000
    • ‏2013-04-02T13:26:18Z
    Unfortunately, you cannot.
    You can only set the other way around, which is the other sub-tree that you want to include in the replication process, even though you may have many of them.

    Rgds. YN.
    I do not have a working replication setup to test this out and I am not able to recall it very well, but I have achieved this in the past by (I guess) adding the auxiliary objectclass "ibm-replicationContext" to the root of the subtree that I do not want to be replicated. For example you have your peer-peer replication setup for o=ibm,c=us, and you want to restrict the subtree ou=XYZ, o=ibm,c=us from being replicated. Try adding the objectclass "ibm-replicationContext" to ou=XYZ,o=ibm,c=us.
  • yn2000
    yn2000
    1086 Posts

    Re: TDS 6.2 Peer Replication Issue/Question

    ‏2013-04-02T14:54:48Z  
    I do not have a working replication setup to test this out and I am not able to recall it very well, but I have achieved this in the past by (I guess) adding the auxiliary objectclass "ibm-replicationContext" to the root of the subtree that I do not want to be replicated. For example you have your peer-peer replication setup for o=ibm,c=us, and you want to restrict the subtree ou=XYZ, o=ibm,c=us from being replicated. Try adding the objectclass "ibm-replicationContext" to ou=XYZ,o=ibm,c=us.
    Quote from: http://www.ibm.com/developerworks/tivoli/library/t-tdsrepl/
    "Replication context: This is the entry for the subtree that is to be replicated..."
    It is not for 'excluded' subtree.

    Rgds. YN.
  • bmatteso
    bmatteso
    108 Posts

    Re: TDS 6.2 Peer Replication Issue/Question

    ‏2013-04-02T15:02:40Z  
    I do not have a working replication setup to test this out and I am not able to recall it very well, but I have achieved this in the past by (I guess) adding the auxiliary objectclass "ibm-replicationContext" to the root of the subtree that I do not want to be replicated. For example you have your peer-peer replication setup for o=ibm,c=us, and you want to restrict the subtree ou=XYZ, o=ibm,c=us from being replicated. Try adding the objectclass "ibm-replicationContext" to ou=XYZ,o=ibm,c=us.
    Interesting isolution. If this worked then it's because replication believes that subtree replication is setup on this lower subtree and this subtree will be replicated via a separate agreement. But since you're just adding the objectclass and presumably no actual agreements, the replication never occurs.
  • yn2000
    yn2000
    1086 Posts

    Re: TDS 6.2 Peer Replication Issue/Question

    ‏2013-04-02T15:12:29Z  
    • yn2000
    • ‏2013-04-02T14:54:48Z
    Quote from: http://www.ibm.com/developerworks/tivoli/library/t-tdsrepl/
    "Replication context: This is the entry for the subtree that is to be replicated..."
    It is not for 'excluded' subtree.

    Rgds. YN.
    You know what, it is doable.

    You use the replication context in the subtree, which overwrite the parent tree, but you just don't configure the replication topology to anywhere else.

    It is the anti-purpose, but it is a brilliant idea.

    Thanks. YN.
  • SystemAdmin
    SystemAdmin
    9855 Posts

    Re: TDS 6.2 Peer Replication Issue/Question

    ‏2013-04-03T06:49:55Z  
    • yn2000
    • ‏2013-04-02T15:12:29Z
    You know what, it is doable.

    You use the replication context in the subtree, which overwrite the parent tree, but you just don't configure the replication topology to anywhere else.

    It is the anti-purpose, but it is a brilliant idea.

    Thanks. YN.
    IIRC the ability to broaden the replication agreements was discussed in this very forum a while ago with the recommendation to create a PMR.

    If such an RFE is implemented that may break this workaround very hard :-)

    And I am sure Ben can tell us that this way of suppressing replication is definitely not within support.

    I would be very concerned to implement this kind of things - in my view replication should do what it is supposed to do - i.e. keeping a number of ldap servers in synch. Also There may be problem with the toolset of TDS e.g. idsldapdiff may not take this into account and hence may "fix" the problem. Also a switch from replica to master may be rather catastrophic...

    I would be glad to understand the underlying requirement - if something should not be visible it should be handled by applying ACLs. Also the proxy server could be the solution if the problem is to "split" the ldap tree.

    Regards
    Franz Wolfhagen
  • bmatteso
    bmatteso
    108 Posts

    Re: TDS 6.2 Peer Replication Issue/Question

    ‏2013-04-03T23:40:39Z  
    IIRC the ability to broaden the replication agreements was discussed in this very forum a while ago with the recommendation to create a PMR.

    If such an RFE is implemented that may break this workaround very hard :-)

    And I am sure Ben can tell us that this way of suppressing replication is definitely not within support.

    I would be very concerned to implement this kind of things - in my view replication should do what it is supposed to do - i.e. keeping a number of ldap servers in synch. Also There may be problem with the toolset of TDS e.g. idsldapdiff may not take this into account and hence may "fix" the problem. Also a switch from replica to master may be rather catastrophic...

    I would be glad to understand the underlying requirement - if something should not be visible it should be handled by applying ACLs. Also the proxy server could be the solution if the problem is to "split" the ldap tree.

    Regards
    Franz Wolfhagen
    Wanted to look into this today, but was just too busy with pmr work. Maybe tomorrow, if I have time. This behavior seems unintentional, though, so doubt that it's officially supported.

    Ben
  • SystemAdmin
    SystemAdmin
    9855 Posts

    Re: TDS 6.2 Peer Replication Issue/Question

    ‏2013-04-04T10:13:11Z  
    • bmatteso
    • ‏2013-04-03T23:40:39Z
    Wanted to look into this today, but was just too busy with pmr work. Maybe tomorrow, if I have time. This behavior seems unintentional, though, so doubt that it's officially supported.

    Ben
    Thank you guys for your useful and constructive statements.
    But I guess this is an issue IBM need to think about. Since this is a customer request.
    The problem is the customer is not considering put an ACL on the affected sub-tree for good reasons: The LDAP data is accessible by many applications with diverse technical accounts it will take a huge management effort to elaborate an appropriate filter which every time need to be updated it there are new changes.
    I am working now about more than 15 years with LDAP and Directory services. I am really wondering this use case is not supported...

    Cheers, Emile
  • SystemAdmin
    SystemAdmin
    9855 Posts

    Re: TDS 6.2 Peer Replication Issue/Question

    ‏2013-04-04T10:35:45Z  
    Thank you guys for your useful and constructive statements.
    But I guess this is an issue IBM need to think about. Since this is a customer request.
    The problem is the customer is not considering put an ACL on the affected sub-tree for good reasons: The LDAP data is accessible by many applications with diverse technical accounts it will take a huge management effort to elaborate an appropriate filter which every time need to be updated it there are new changes.
    I am working now about more than 15 years with LDAP and Directory services. I am really wondering this use case is not supported...

    Cheers, Emile
    I beg to differ....

    Customer requirements are important - but the IMPLEMENTATION of a requirement has to follow the overall architecture - else things starts to break in other places.

    The requirement is clearly solvable by using the ACLs that are actually there for the same purpose....

    That a tree design has a certain effect on how the security is performed should after 15 years really not be a surprise - this is (and has always been) an important design point when making tree structures.

    Replication is a different thing used for a different thing - namely making a clustered structure available - I would be very worried is we start mixing access to data (security) with replication (infrastructure).

    Now - that said - have you looked into proxy server - it has some functionality that could make it possible for you to split out the actual tree (the one you do not want to replicate) into a separate ldap instance - having a proxy server up front would make that invisible to all requests going through the proxy server. See http://pic.dhe.ibm.com/infocenter/tivihelp/v2r1/topic/com.ibm.IBMDS.doc/admin_gd282.htm?path=7_4_4_11#distribdir

    Also ITDS comes with TDI to help you if you want do advanced things that is not possible with the standard utilities - i.e. making a "replication" like this - it is a "relatively" simple task to setup - but making it enterprise grade is a different story (HA,fail-over etc.).

    A last thing - if you want to have your requirement feed to the IBM Product Management team you should created a Request For Enhancement (RFE) here - but remember that is no guarantee that it is ever going to be solved : http://www.ibm.com/developerworks/rfe/?BRAND_ID=90

    HTH

    Regards
    Franz Wolfhagen
  • SystemAdmin
    SystemAdmin
    9855 Posts

    Re: TDS 6.2 Peer Replication Issue/Question

    ‏2013-04-04T10:53:18Z  
    I beg to differ....

    Customer requirements are important - but the IMPLEMENTATION of a requirement has to follow the overall architecture - else things starts to break in other places.

    The requirement is clearly solvable by using the ACLs that are actually there for the same purpose....

    That a tree design has a certain effect on how the security is performed should after 15 years really not be a surprise - this is (and has always been) an important design point when making tree structures.

    Replication is a different thing used for a different thing - namely making a clustered structure available - I would be very worried is we start mixing access to data (security) with replication (infrastructure).

    Now - that said - have you looked into proxy server - it has some functionality that could make it possible for you to split out the actual tree (the one you do not want to replicate) into a separate ldap instance - having a proxy server up front would make that invisible to all requests going through the proxy server. See http://pic.dhe.ibm.com/infocenter/tivihelp/v2r1/topic/com.ibm.IBMDS.doc/admin_gd282.htm?path=7_4_4_11#distribdir

    Also ITDS comes with TDI to help you if you want do advanced things that is not possible with the standard utilities - i.e. making a "replication" like this - it is a "relatively" simple task to setup - but making it enterprise grade is a different story (HA,fail-over etc.).

    A last thing - if you want to have your requirement feed to the IBM Product Management team you should created a Request For Enhancement (RFE) here - but remember that is no guarantee that it is ever going to be solved : http://www.ibm.com/developerworks/rfe/?BRAND_ID=90

    HTH

    Regards
    Franz Wolfhagen
    -oh - I just realized that the RFE link was the "old" Tivoli link - the Security Systems RFE should go here : http://www.ibm.com/developerworks/rfe/?BRAND_ID=301

    Sorry for the invconvience

    regards
    Franz Wolfhagen
  • bmatteso
    bmatteso
    108 Posts

    Re: TDS 6.2 Peer Replication Issue/Question

    ‏2013-04-04T17:07:39Z  
    -oh - I just realized that the RFE link was the "old" Tivoli link - the Security Systems RFE should go here : http://www.ibm.com/developerworks/rfe/?BRAND_ID=301

    Sorry for the invconvience

    regards
    Franz Wolfhagen
    Thanks for including the RFE link Franz. If this is an important customer request, there really should be either an RFE opened or a pmr opened. As it happens, I believe I will be able to find out if this behavior would be supported or not today. More to come soon ...
  • SystemAdmin
    SystemAdmin
    9855 Posts

    Re: TDS 6.2 Peer Replication Issue/Question

    ‏2013-04-04T17:25:10Z  
    • bmatteso
    • ‏2013-04-04T17:07:39Z
    Thanks for including the RFE link Franz. If this is an important customer request, there really should be either an RFE opened or a pmr opened. As it happens, I believe I will be able to find out if this behavior would be supported or not today. More to come soon ...
    Well - IF this is a VERY important requirement then a customer can escalate it up the IBM SWG management chain - we as simple IBM workers are not part of that :-)

    But as I see it the requirement CAN be solved using supported techniques - yes it requires some work - TANSTAAFL applies - but I am just starting getting worried when a requirement is transformed into a specific method of implementing it. Transforming requirements to implementations are my home turf (being an IAM implementer) and challenging a requirement is a vital part of the process.

    Best solutions are actually coming when we are disagreeing with the customer when we start :-)

    That is also the reason I stick out my neck here - I do not like when I see something I believe that will hurt a customer on the long term - on the other side it does not mean that I am right...

    Regards
    Franz Wolfhagen