robots.txt blocking urls, how to stop this?

monique062579 · #1 () 12-19-2007, 02:09 AM

There are about 300 urls on my site that are being blocked by robot.txt, but I did not put any robot.txt files on my site at all so why is this happening? How can I stop this because the urls that are being blocked are urls that go right to my product description pages. I want these accessible for the bots to crawl. Any suggestions on how to go about this?

GonzaloGil · #2 () 12-19-2007, 02:14 AM

You could login via FTP and delete the robots.txt
However, I suggest you don't. The reason is the default robots.txt blocks the search engines from accessing files such as "add_cart.as" which should not be indexed, as well as others.

monique062579 · #3 () 12-19-2007, 10:18 PM

I am just worried that this is causing my site not to get indexed well. Do you think that it is doing that?