Question & Answer
Question
This technote discusses one way to deploy a robots.txt in WebSphere.
Cause
robots.txt can be used to prevent a search engine web crawler from accessing certain directories or files.
robots.txt are typically deployed in front-end proxies or web servers (like IBM HTTP Server) in production environments.
However, you might occasionally want to deploy a robots.txt directly on WebSphere itself, like in a single server development environment.
Answer
Every "application" in WebSphere needs to be deployed as an EAR (enterprise application). You cannot just put a file in a certain directory (like in Apache) in order to deploy it.
You can create an EAR, which contains a WAR (web application) that is mapped to context root / (that is, context root "forward slash"). The WAR can contain the robots.txt file.
The EAR should be installed in the particular JVM, probably using virtual host default_host. After starting the EAR, you should be able to successfully access http://<hostname>:<port>/robots.txt.
Related Information
Was this topic helpful?
Document Information
More support for:
WebSphere Portal
Software version:
6.1
Operating system(s):
AIX, HP-UX, Linux, Solaris, Windows
Document number:
406703
Modified date:
03 December 2021
UID
swg21430544