Indexing stops

[ricotrutt]

Hello everyobdy,

I've got a problem with the sitemap generator : it stops in the middle of indexing my 380K-pages website. Actually, it works perfectly, then suddently stops in the middle of indexing, never at the same number of pages indexed. Did you encounter this problem ? Do you have a solution for this ?

Best regards

[V]

Sorry that you got this problem. The tool was never designed for such big load and that's why it doesn't work well with big sites. In general, it will get the whole memory full and will stop and/or crash. I am working on this, trying to create the next version to support large sites, but it is not yet ready. A part of that, it is always better to use some server-side sitemap generation if your site is database-driven -- a lot easier and faster (if possible in the given case).

[ricotrutt]

Please don't be sorry, you already propose a very interesting source for free.

I don't know what tool i could use as "server side sitemap generation", do you have any idea of what i could use ?

Best regards

[V]

Well, there is no general solution here -- it all depends on your site. If the site is based on some CMS (like WordPress, Joomla! etc) -- there are plugins or modules that could do the job. If the site is custom-made, it comes to programming (php, asp etc.) and extracting data from the database (mysql, postgre...) So, it all depends on the site itself. If you don't know is your site database driven and/or what server-side technology it uses, there is no much I can do. However, if you want you can send me a PM with your site URL and I will try to see if there is simple solution.

[V]

It looks like your site is Drupal based -- so you should use some sitemap generation module for Drupal (like this: http://drupal.org/project/site_map)
The ecommerce engine (Ubercart) seems to support xml sitemap too: http://www.ubercart.org/docs/user/336/xml_sitemap

But for specific instructions on the given site (setup) you should refer the respective software developers (or the webmaster who built your site)

[ricotrutt]

Ok, i will try that. Thanks for your answer !