Answer

The REMOTE_ADDR value in the request is an indexed field that can be used for searching and merging sessions by client IP address. However if a network device has over-written this value due to network address translation (NAT) the search value of this field is removed. In most cases the network device or service will preserve the true client address in another header which can then be used to correct the REMOTE_ADDR value. The most common headers for storing the original address are the "client true IP" and "x-forwarded-for" headers.

The examples provided here show how a Privacy Reqset rule can be used for overlaying this value in either the PCA or Transport Service.

Copying an IP address from HTTP_TRUE_CLIENT_IP

In some cases load balancers or network boundary caching service (Akamai etc.) will overlay the address in REMOTE_ADDR to be the address of the distribution server not the actual address of the web site user. If the service provider has added a ?true? IP field this can be overlaid onto REMOTE_ADDR to correct it.

HTTP_TRUE_CLIENT_IP ? REMOTE_ADDR example:

[CorrectRemote_Addr]

Action=ReqSet

ReqSetSection=env

ReqSetField=REMOTE_ADDR

Section=env

StartPattern=\nHTTP_TRUE_CLIENT_IP=

EndPattern=\r

Note:

If the source field ?HTTP_TRUE_CLIENT_IP? does not exist no action is applied.
Therefore when there is a mix of traffic and only some IPs require correction
the rule will do no harm for non-forwarded sessions.

Copying an IP address from HTTP_X_FORWARDED_FOR

The HTTP_X_FORWARDED_FOR request header is added by network devices to preserve the ?real? IP address
when network address translation will replace that address. The HTTP_X_FORWARDED_FOR header can take two forms:

X-Forwarded-For: client proxy1 proxy2

or:

X-Forwarded-For: client

..

X-Forwarded-For: proxy1

..

X-Forwarded-For: proxy2

(Note that in Tealeaf capture data these headers appear as ?HTTP_X_FORWARDED_FOR?)

While the first form is standard in some cases network devices may simply add another header resulting in multiple headers.The second format is also an acceptable standard so long as the multiple headers can still be ?rolled up? into a combined replacement. The first occurrence of the header is the client true IP in that case.


HTTP_X_FORWARDED_FOR ? REMOTE_ADDR example:

[test1]
Action=ReqSet

ReqSetSection=env

ReqSetField=REMOTE_ADDR

Section=env

StartPattern=\nHTTP_X_FORWARDED_FOR=

EndPattern=\r

The above example always catches the first occurrence of this header value. It would not be appropriate to use this rule when the x-forward value contains a list of addresses as this would copy all of them.

The next example will catch the last occurrence. This example is provided for illustration purposes only as it the last IP address is a device IP not the original browser IP:

[test1]

Action=ReqSet

ReqSetSection=env

ReqSetField=REMOTE_ADDR

Section=env

Field=HTTP_X_FORWARDED_FOR

The Field option gathers the contents of the header but in practice overlays itself until it finds the last occurrence. While a ?RepeatCount=1? option might seem to fix this that option is only for start patterns not for the Field option. Further for ReqSet ReqAppend and ReqDelete when a start pattern is used ?RepeatCount=1? is already enforced. As in the previous example it would not be appropriate to use this rule when the x-forward value contains a list of addresses as this would copy all of them.

Both of the above examples catch the entire header value to the end of the line...but for REMOTE_ADDR we want only the first address. What if some requests have multiple HTTP_X_FORWARDED_FOR headers while others have the combined format?

Action=ReqSet

ReqSetSection=env

ReqSetField=REMOTE_ADDR

Section=env

StartPatternRe=\nHTTP_X_FORWARDED_FOR=([\w|\.]+)

ReqSetResult={g1}

Inclusive=True

The above rule will find the first occurrence of this header and return only the first IP address from it regardless of whether there is one address or a comma delimited list. This is the most reliable approach.