JCrawler Forum Topic: Jcrawler exceptions

JCrawler Forum Topic: Jcrawler exceptions http://www.pixelschieber.ch/forum/ Help forums for JCrawler en Fri, 30 Jul 2010 23:20:40 +0000 Metal Wing on "Jcrawler exceptions" http://www.pixelschieber.ch/forum/topic/jcrawler-exceptions#post-416 Tue, 08 Dec 2009 05:32:33 +0000 Metal Wing 416@http://www.pixelschieber.ch/forum/ You, Sir, are a genius ~ Thank you! Worked perfectly. P.S. Yes, I was using the latest build. patrick on "Jcrawler exceptions" http://www.pixelschieber.ch/forum/topic/jcrawler-exceptions#post-415 Mon, 07 Dec 2009 22:47:22 +0000 patrick 415@http://www.pixelschieber.ch/forum/ hey, just insert "events" and on the next line "com_jevents" in the exclude field, then the crawler should ignore the jevents-component. patrick ps:did you try the latest version: <a href="http://www.pixelschieber.ch/com_jcrawler_latestbuild.zip" rel="nofollow">http://www.pixelschieber.ch/com_jcrawler_latestbuild.zip</a> Metal Wing on "Jcrawler exceptions" http://www.pixelschieber.ch/forum/topic/jcrawler-exceptions#post-414 Mon, 07 Dec 2009 21:33:30 +0000 Metal Wing 414@http://www.pixelschieber.ch/forum/ Hi, I have not been able to get JCrawler to avoid crawling certain areas of my site. I have Joomla SEF URLs enabled and also have JEvents installed, so when JCrawlers begins crawling it picks up every single calendar url for the past and future 10 years lol There are errors like # Curl error on url <a href="http://www.animewave.net/events/day.listevents/2009/11/06/-.html" rel="nofollow">http://www.animewave.net/events/day.listevents/2009/11/06/-.html</a>: Connection time-out # Curl error on url <a href="http://www.animewave.net/events/day.listevents/2009/11/07/-.html" rel="nofollow">http://www.animewave.net/events/day.listevents/2009/11/07/-.html</a>: Connection time-out # Curl error on url <a href="http://www.animewave.net/events/week.listevents/2009/11/8/-.html" rel="nofollow">http://www.animewave.net/events/week.listevents/2009/11/8/-.html</a>: Connection time-out And URLs that were indexed are <a href="http://www.animewave.net/events/day.listevents/2009/12/15/-.html" rel="nofollow">http://www.animewave.net/events/day.listevents/2009/12/15/-.html</a> <a href="http://www.animewave.net/events/day.listevents/2009/12/16/-.html" rel="nofollow">http://www.animewave.net/events/day.listevents/2009/12/16/-.html</a> What I haven't been able to do is to make it so JCrawler COMPLETELY avoids the calendar extension. I have tried telling it to avoid the SEF link <a href="http://www.animewave.net/events/" rel="nofollow">http://www.animewave.net/events/</a> I have tried telling it to avoid a component link <a href="http://animewave.net/index.php?option=com_jevents&task=month.calendar" rel="nofollow">http://animewave.net/index.php?option=com_jevents&task=month.calendar</a> Any help? Because I really would love to index my site with JCrawler, but I really don't want 10000 URLs just for the calendar...