Topic
  • 5 replies
  • Latest Post - ‏2013-02-13T19:51:03Z by warrenm1
SystemAdmin
SystemAdmin
403 Posts

Pinned topic Long scan times and tools for analyzing traffic logs?

‏2013-02-06T03:00:48Z |
Hi --

I'm working with AppScan Enterprise. Currently, I'm dealing with an issue where the scan log is reporting "Scan task taking more time than expected". To source the problem, I've turned on verbose logging and traffic logging. I've been going through the logs for the better part of two days, but have been unable to optimize the scan.

In the Item Log, I've located messages that look like...
"A worker thread is taking more than expected to process a task" "WorkerThread Monitor"
...
" Thread ID: 1234" "WorkerThread Monitor" ""
Processed: 139 task(s)." "WorkerThread Monitor" ""
" CurTask is taking: 00:40:14.1250136" "WorkerThread Monitor" ""
"Task name: Security Entity: ParamName | SOME_PARAMETER_NAME |

By subtracting the duration from the timestamp, I'm able to trace that back to an area in the Item Log where I believe the task starts. That looks like...

information "02/04/2013 15:14:29.066" AgentHost.exe 1515 1515 2020 "Scan Engine" 0x0 "XRule 'probeXSS1' on 'Security Entity: ParamName | SOME_PARAMETER_NAME | SOME_URL ' - rule is running" "XsrExecutionEnvironment.RunRule" ""
What's odd, is although probeXSS1 may say that it's "running" at 15:14:29, I don't actually see it reflected in the traffic logs until 10+ minutes later. I'm assuming at this point... the message "Scan task taking more time than expected" is likely referring to a huge class of tests and not just one specific, long running, test.

Further, from the traffic log, it's clear that tests are still being executed.

Is there any better way to isolate the source(s) of this problem? At this point, I figured I would just write a script to analyze the time it takes for the application to respond to each request and look for the tests that are taking significant amounts of time to run. Do any utilities (or reports within AppScan Enterprise) already exist for this purpose? Something that could provide the following information would be very useful:

(1) Average request/response time.
(2) Longest request/response wait times.
(3) Security test execution time.
Any information would be very very helpful.
Updated on 2013-02-13T19:51:03Z at 2013-02-13T19:51:03Z by warrenm1
  • SystemAdmin
    SystemAdmin
    403 Posts

    Re: Long scan times and tools for analyzing traffic logs?

    ‏2013-02-06T04:48:52Z  
    For anyone else having trouble with this, the most effective approach I've found is setting the number of allowed threads/connections to 1 and searching the Item Log file for "Executing". This should provide an ordered list of rules that are executed. By looking through the timestamps, it should be fairly clear where the majority of time is being spent. If you start seeing areas where it's taking more than 5 seconds to go from one rule to the next (in my case, I was seeing delays of over 20 seconds), it would be worth investigating the traffic log for those rules and see what's going on.

    If anyone else has a more effective approach, I would be interested in hearing it.

    Somewhat related, if anyone knows if it's possible to disable logging the response body (when you logging traffic), I would be excited to know how. Often, when troubleshooting through the traffic log -- only the request header, request body, and response header are important (IMO). In my situation, this would reduce an 800MB log file by probably 90% and make it much more manageable (and reduce the potential of logging sensitive application information).
  • SystemAdmin
    SystemAdmin
    403 Posts

    Re: Long scan times and tools for analyzing traffic logs?

    ‏2013-02-09T01:24:52Z  
    That notice may not be a problem if the thread does complete.
    Setting your threads to 1 will make it easier to see what is happening in the logs and may make the notice go away.

    This might be due to your scan being too large. There are a number of knowledge articles on this topic.
  • SystemAdmin
    SystemAdmin
    403 Posts

    Re: Long scan times and tools for analyzing traffic logs?

    ‏2013-02-11T20:54:36Z  
    That notice may not be a problem if the thread does complete.
    Setting your threads to 1 will make it easier to see what is happening in the logs and may make the notice go away.

    This might be due to your scan being too large. There are a number of knowledge articles on this topic.
    > 708B_ken_reed wrote:
    > That notice may not be a problem if the thread does complete.
    > Setting your threads to 1 will make it easier to see what is happening in the logs and may make the notice go away.
    >
    > This might be due to your scan being too large. There are a number of knowledge articles on this topic.
    I've set the number of permitted threads/connections to 1 and have reviewed the logs in detail. The issue, I discovered, is due to a high number of HTTP requests being made in between tests. When AppScan starts testing certain URL parameters, there are 100+ HTTP requests that occur in between any two tests. As AppScan re-requests large amounts of static JavaScript content, a 16 second delay occurs.

    I posted this in a separate thread, but I will ask it again here -- my delay would be reduced by 50% if I could tell AppScan to cache JavaScript content. Is there any way for me to do this?

    Again, this is a manually recorded scan that I am performing (it does not use auto-spider).
  • SystemAdmin
    SystemAdmin
    403 Posts

    Re: Long scan times and tools for analyzing traffic logs?

    ‏2013-02-11T21:20:41Z  
    That notice may not be a problem if the thread does complete.
    Setting your threads to 1 will make it easier to see what is happening in the logs and may make the notice go away.

    This might be due to your scan being too large. There are a number of knowledge articles on this topic.
    > This might be due to your scan being too large. There are a number of knowledge articles on this topic.
    FWIW, I did go through the articles you listed in a separate thread and can confirm that my situation does not match any of the scenarios that were listed. The site is very (very) small and the only rules that are currently being tested are the 'Vital Few'.
  • warrenm1
    warrenm1
    224 Posts

    Re: Long scan times and tools for analyzing traffic logs?

    ‏2013-02-13T19:51:03Z  
    > 708B_ken_reed wrote:
    > That notice may not be a problem if the thread does complete.
    > Setting your threads to 1 will make it easier to see what is happening in the logs and may make the notice go away.
    >
    > This might be due to your scan being too large. There are a number of knowledge articles on this topic.
    I've set the number of permitted threads/connections to 1 and have reviewed the logs in detail. The issue, I discovered, is due to a high number of HTTP requests being made in between tests. When AppScan starts testing certain URL parameters, there are 100+ HTTP requests that occur in between any two tests. As AppScan re-requests large amounts of static JavaScript content, a 16 second delay occurs.

    I posted this in a separate thread, but I will ask it again here -- my delay would be reduced by 50% if I could tell AppScan to cache JavaScript content. Is there any way for me to do this?

    Again, this is a manually recorded scan that I am performing (it does not use auto-spider).
    Hi

    If you have the scan set to only test manual explore traffic and you are seeing this many requests there are a few possibilities. The Most likely is the You Have Javascript Execution enabled and by executing the javascript in prior responses more requests are generated. Javascript can also be resource intensive and take longer to execute than straight up http requests which frequently lead to the 'scan task taking longer...' type errors.

    The only real value of enabling Javascript execution ins in AJAX style apps when you are performing an automatic crawl, it helps the discovery of urls, if you are testing manual explore only I would advise turning it off for best results.

    Other possible reasons for multiple requests in between tests is if there is something wrong in your in-session settings generating frequent logins, for that type of analysis you'd need to log a pmr with Appscan Support so the log/settings can be analyzed in detail.

    Hope this helps,