sitemap for large website

[pornphase]

how do you go about creating sitemaps for large websites. do you link to other xml files from the main xml file? or do you create a static html page with several diffrent xml files? id like to know your thoughts.

thanks,
Porn Phase!

[johnclark]

Something like this http://www.pornhub.com/sitemap

[InfiniteTech]

Quite a few applications are available here:

https://code.google.com/p/sitemap-generators/wiki/SitemapGenerators

Both server side and client side software can crawl through your website and create a useful sitemap.

[pornphase]

johnclark, that is not a xml sitemap though, ive been reading that google likes xml for sitemaps. im guessing this isnt so true?

also how are people getting the "sub pages links" on the google search results. for example google porn hub... you will see categories, top rated, ect ect but they are not in the sitemap for pornhub.

[InfiniteTech]

also how are people getting the "sub pages links" on the google search results. for example google porn hub... you will see categories, top rated, ect ect but they are not in the sitemap for pornhub.

As far as I'm aware that is Google being intelligent. Nothing to do with sitemaps directly.

Although a hierarchy in the XML sitemap you generate might help.

[pornphase]

my site is in full javascript so a sitemap is really needed for geting indexed.

im doing both... thanks guys ;]

[johnclark]

Also try something like this:

Sitemap.xml
-Posts-sitemap.xml
--Posts sorted by month
---The post
-Pages-Sitemap.xml
-Tags-sitemap.xml
-Categories-sitemap.xml
-Parent Categories
--The Categories

Just an example.

[dunhill]

Also try something like this:

Sitemap.xml
-Posts-sitemap.xml
--Posts sorted by month
---The post
-Pages-Sitemap.xml
-Tags-sitemap.xml
-Categories-sitemap.xml
-Parent Categories
--The Categories

Just an example.

This example would contain duplicate webpages and will delay the indexing I think.
OP will need a sitemap index (https://support.google.com/webmaster...er/71453?hl=en)

[dobry_den]

Read this: support.google.com/webmasters/answer/183668?hl=en

I wouldn't even bother with XML unless you really do have a massive set of urls that can't all fit in a single file (a single sitemap file should have no more than 50k urls) and you have a wide variety of pages that get updated at vastly different times. XML sitemaps are only useful because they let you specify priority and last-modified stamps for each url.

Otherwise I'd just go for the "URL list" style sitemap which is a file with a url on each line.

If you need to enumerate your urls across multiple files, then you create a sitemap index file (support.google.com/webmasters/answer/71453) and submit that instead.

(I'm not allowed to post urls as a noob)