Configuration options that may be applied on a per-URL basis.
Attributes
- added (May only be: added) - Flag when, if present, indicates that a
particular curl-option was added by the crawl conditions.
- name (Any of: proxy, proxy-port, interface, user-password,
proxy-user-password, http-auth, http-proxy-auth, referer, http-version, user-agent
[Default: VSE/1.0], header, cookie, timeout [Integer Default: 600], connect-timeout
[Integer Default: 30], ftplistonly [Boolean Default: false], max-hops [Integer Default:
15], max-bytes [xs:long Default: 1000000000], max-data-size [xs:long Default:
100000000], max-redirs [Integer Default: 10], internal-max-redirs [Integer Default: 0],
max-slashes [Integer Default: -1], max-url-len [Integer Default: -1],
encoding-replacements, ignore-reported-encoding [Boolean Default: false],
default-encoding, default-content-type, forced-content-type, robots, case-normalize,
delay [Integer Default: 100], n-concurrent-requests [Integer Default: 1], error-expires
[Integer Default: 604800], uncrawled-expires [Integer Default: 0], expires [Integer
Default: 0], force-recrawl, log [Boolean Default: true], disable-log [Boolean Default:
false], disable-graph [Boolean Default: false], graph [Boolean Default: true],
filter-exact-duplicates [Boolean Default: false], ignore-robots [Boolean Default:
false], disable-dns-resolution [Boolean Default: false], converter-max-cpu [Integer
Default: 600], converter-max-memory [Integer Default: 512], converter-max-elapsed
[Integer Default: 600], post, method [Default: GET], full-logging, refined-logging,
minimal-logging, url-logging [Default: refined-logging], default-allow [Default:
disallow], authorization-required [Boolean Default: false], authorization-url,
default-acl, enable-acl [Boolean Default: true], acl-user-prefix, priority [Integer
Default: 0], enable-fetch-compression [Boolean Default: true], ssl-version [Text
Default: Any], ssl-cert, ssl-cert-type, ssl-key, ssl-key-type, ssl-key-password,
ssl-ca-cert, ssl-verify-peer, converter-fallback-type [Text Default: vivisimo/fallback],
crawler-fallback-type [Text Default: vivisimo/crawler-error], test-conversion [Integer
Default: 0], test-conversion-id [Integer Default: -1], test-conversion-debug-id,
immediate-persistence, lazy-persistence, enqueue-persistence [Default:
immediate-persistence], max-export-size [Decimal number Default: 0], purge-input-xml
[Boolean Default: false], enqueue-input-threshold [Integer Default: 100000],
purge-complete-xml [Boolean Default: false], purge-all-xml [Boolean Default: false],
form-authentication-steps [Text], form-authentication-root [Text],
form-authentication-debug [Boolean Default: false])
- value (Text)