archive.org -- robots.txt

Collapse
X
Collapse
 
  • Page of 1
  • Time
  • Show
Clear All
new posts
Previous template Next
  • d-null
    . . .
    • Apr 2007
    • 13724
    #1

    archive.org -- robots.txt

    Is there a reason or advantage for using robots.txt to disallow archive.org from archiving your sites?

    I noticed that gfy disallows it, so made me curious one way or another

    any thoughts?

    __________________
    Looking for a custom TUBE SCRIPT that supports massive traffic, load balancing, billing support, and h264 encoding? Hit up Konrad!
    Looking for designs for your websites or custom tubesite design? Hit up Zuzana Designs
    Check out the #1 WordPress SEO Plugin: CyberSEO Suite
    Tags: None
    • d-null
      . . .
      • Apr 2007
      • 13724
      #2
      will try a bump hoping for an opinion

      __________________
      Looking for a custom TUBE SCRIPT that supports massive traffic, load balancing, billing support, and h264 encoding? Hit up Konrad!
      Looking for designs for your websites or custom tubesite design? Hit up Zuzana Designs
      Check out the #1 WordPress SEO Plugin: CyberSEO Suite

        Comment

        • fpforum
          Confirmed User
          • Dec 2006
          • 1045
          #3
          maybe they just don't want there page's to be stored there..some people also like to block things like there image's so they aren't indexed in google images and what not..Personally I don't see any advantage or disadvantage to it..If you block it, noone will be able to see your cached page's in the wayback machine..if you don't disalow it, they will..not like it's going to effect your rankings in the SERPS or anything..

          hope this helps!

          Email: strikedata |@| gmail.com Skype: mistercashman

            Comment

            • Subtle
              Confirmed User
              • Jun 2007
              • 2005
              #4
              Any more opinions?

              ----- Blog Themes for $49 -----
              ------ INSTANT DELIVERY ------

                Comment

                Previous template Next
                Working...