Browscap file version notes

Answer

What are the considerations for updating the Browscap file?

From time to time the file format or contents has changed and introduced parsing problems.
These have historically produced session agent failures or incorrect agent string matching.

For more information on how Browscap files are installed and managed see article 5377-16440

File size (A performance concern)

Historically the Browscap.csv file was small and of stable size but recent versions have grown very quickly:

5008 =???? 4K lines 1.3MB (July 2012)
5029 = ? 48K lines 4.4MB (May 2014)
5032 =?? 57K lines 17MB (August 2014)
5034 =?? 71K lines 20MB (Oct 2014)
5036 =?? 78K lines 23MB (Nov 2014)
6001 = 109K lines 60MB (Apr 2015)
?

The best practice for good performance is to set the TLTRef cache size to match the number of lines in the file:

MaxCacheSize=2000 (default)

MaxCacheSize=100000 (for about 100K lines)

To adjust the MaxCacheSize edited the TLTRef session agent configuration in your transport service either via TMS or in the \Tealeaf\TealeafCaptureSocket.cfg file. The change is effected upon transport service restart.

When this cache size is too small frequent file system access will lower the processing rate and the pipeline will experience "pipeline spooling". However increasing the memory footprint of the TLTRef session agent may make it impossible to place in child pipelines of an HBR due to limited access to memory in a 32 bit process. The 9.0.2 version provides multi-process pipelines to improve this situation.

Browscap file versions

From time to time the file format or contents has changed and introduced parsing problems.
These have historically produced session agent failures or incorrect agent string matching.

5033+ (Sept 2014)

Prior patching for version 5032 works well with that version but the next Browscap file version released5033
introduced yet another change in formatting (a dropped column) and this produces new parsing errors.

Direct patches for versions 8.7 and 8.8 are available via support requests and the fix is included in 8.7 Fix Pack 5
and later fix packs. The patch has been provided and tests well with file version 5036.

An example agent string that is recognized in 5036 but not 5032:

Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML like Gecko) Chrome/38.0.2125.111 Safari/537.36

5029+ (May 2014)

This file version did not trigger a warning or session agent failure but did trigger incorrect user agent recognition.
A common sign of the problem was for valid agent strings (IE etc.) to be tagged as "Fake Browser9.0"
and for the traffic type to be set to Bot. This is a particularly large problem when data drop rules are dropping bot traffic.

The last public Browscap release version prior to this change was version 5026 and reverting to that version
removes this error in agent recognition.

An August 6 fix was found to be incomplete. In early September 2014 much improved parsing was released:?

- 8.8.0.8895_TL_CXUpgrade_FixPack3 (a full updater) ?
- 8.8.0.8897_TL_CXPatchForBrowscap5032 (an "interim" drop-in fix)
- 8.7.1.8799_TL_CXPatchForBrowscap5032? (an "interim" drop-in fix)

All three include the same fix set including the user agent revealer and UA matchers.
Testing has gone well with Browscap versions 5030 and 5032 (Aug04).

There is one remaining (known!) problem in that agents with a "syndication" flag are marked as traffic type "Bot".
These are usually email clients and there are about 150 of them.

5000 (May 2012)

This version introduced a transport service error that was visible in TLTRef startup messages in the Windows application log.
The most recent Tealeaf version to require patching for this file format was 7.2 (7296).

Article Reference