#robots text file is used to control the access to web #site files and directories which should not be indexed # User-agent: * # #Block spidering of specific top level directories # #Updated 07-SEPT-06 # # Disallow: /admin