{"id":443,"date":"2007-07-27T14:01:00","date_gmt":"2007-07-27T05:01:00","guid":{"rendered":"https:\/\/iwatakenichi.com\/2007\/07\/yetibot%e3%81%ae%e5%86%8d%e6%9d%a5\/"},"modified":"2007-07-27T14:01:00","modified_gmt":"2007-07-27T05:01:00","slug":"yetibot%e3%81%ae%e5%86%8d%e6%9d%a5","status":"publish","type":"post","link":"https:\/\/iwatakenichi.com\/?p=443","title":{"rendered":"yetibot\u306e\u518d\u6765"},"content":{"rendered":"<p><a href=\"http:\/\/iwatakenichi.blogspot.com\/2007\/07\/yeti-naverbot.html\">\u5148\u65e5\u306e\u8a18\u4e8b<\/a>\u3092\u66f8\u3044\u3066robots.txt\u3092\u8a2d\u5b9a\u3057\u305f\u3042\u3068\u3001\u3071\u3063\u305f\u308a\u3053\u306a\u304f\u306a\u3063\u3066\u3044\u305fyetibot@naver.com\u3055\u3093\u3067\u3059\u304c\u3001robots.txt\u306e\u5236\u9650\u3092\u5916\u3057\u3066\u304a\u3044\u305f\u304b\u3044\u3042\u3063\u3066(?)\u300125\u65e5\u306e\u671d9\u6642\u9803\u304b\u3089\u6d3b\u52d5\u3092\u518d\u958b\u3057\u305f\u3088\u3046\u3067\u3059\u3002<a href=\"http:\/\/saas-sfa.com\/\">\u4f8b\u306e\u30b5\u30a4\u30c8<\/a>\u3078\u306e\u30a2\u30af\u30bb\u30b9\u304c25\u65e5\u306e\u671d9\u6642\u304b\u308926\u65e5\u306e\u671d3\u6642\u307e\u3067\u30674247\u30a2\u30af\u30bb\u30b9\u3067\u3057\u305f\u3002\u305d\u306e\u3046\u3061\u7d04\u534a\u6570\u306e2125\u30a2\u30af\u30bb\u30b9\u304ccss\u30d5\u30a1\u30a4\u30eb\u3067\u3059\u3002\u4f8b\u306b\u3088\u3063\u3066\u3002<\/p>\n<p>\u4eca\u56de\u306f\u3044\u308d\u3044\u308d\u5b9f\u9a13\u3057\u3066\u307f\u305f\u3044\u306e\u3067\u3001\u7d20\u76f4\u306b\u306f\u3058\u304f\u306e\u306f\u3084\u3081\u3066\u3001robots.txt\u306b\u6b21\u306e\u3088\u3046\u306b\u66f8\u3044\u3066\u307f\u307e\u3057\u305f\u3002<\/p>\n<pre><code>\nUser-agent: Yeti\nDisallow: \/css\/\nDisallow: \/img\/\nCrawl-delay: 180\n\nUser-agent: Baidu\nDisallow: \/css\/\nDisallow: \/img\/\nCrawl-delay: 180\n\nUser-agent: *\nDisallow:\n<\/code><\/pre>\n<p>Crawl-delay\u306f<a href=\"http:\/\/www.robotstxt.org\/wc\/norobots.html\">A Standard for Robot Exclusion<\/a>\u306b\u306f\u542b\u307e\u308c\u3066\u3044\u306a\u3044\u3088\u3046\u3067\u3059\u304c\u3001ask.com\u3084Yahoo!\u306e\u30dc\u30c3\u30c8\u306f\u30b5\u30dd\u30fc\u30c8\u3057\u3066\u3044\u308b\u3088\u3046\u3067\u3059\u3002\u53c2\u8003\u6587\u732e[2]\u306b\u3088\u308b\u3068Yeti\u3055\u3093\u306fCrawl-delay\u3092\u7406\u89e3\u3067\u304d\u306a\u3044\u3088\u3046\u3067\u3059\u304c\u3001\u307e\u3042\u3044\u3061\u304a\u3046\u308f\u305f\u3057\u3082\u8a66\u3057\u3066\u307f\u305f\u3044\u306e\u3067\u3001\u66f8\u3044\u3066\u307f\u307e\u3057\u305f\u3002\u3042\u3068\u306fcss\u3092\u6bce\u56de(!)\u8aad\u307f\u306b\u304d\u3066\u3046\u308b\u3055\u3044\u306e\u3067\u3001\u3053\u308c\u3082\u7981\u6b62\u306b\u3002\u30a4\u30e1\u30fc\u30b8\u30d5\u30a1\u30a4\u30eb\u306f\u8aad\u307f\u306b\u304d\u3066\u306f\u3044\u306a\u3044\u3051\u3069\u3001\u5ff5\u306e\u305f\u3081\u3002<\/p>\n<p>\u3064\u3044\u3067\u306b(?)Baidu\u3055\u3093\u3082\u6307\u5b9a\u3057\u3066\u307f\u305f\u3002\u73fe\u5728\u306e\u3068\u3053\u308d\u3001Baiduspider\u3055\u3093\u306f\u30b3\u30f3\u30b9\u30bf\u30f3\u30c8\u306b2\u5206\u306b1\u30a2\u30af\u30bb\u30b9\u30011\u6642\u9593\u306b30\u56dex24\u6642\u9593=720\u30a2\u30af\u30bb\u30b9\u7a0b\u5ea6\u306e\u300c\u826f\u8b58\u7684\u300d\u30a2\u30af\u30bb\u30b9\u3067\u3059\u3002\u3068\u308a\u3042\u3048\u305a\u9593\u9694\u30923\u5206\u306b\u3057\u3066\u3082\u3089\u3048\u308b\u304b\u3069\u3046\u304b\u30c6\u30b9\u30c8\u306e\u305f\u3081\u52a0\u3048\u3066\u307f\u307e\u3057\u305f\u3002<\/p>\n<p>\u3055\u3042\u3069\u3046\u306a\u308b\u304b\u306a\u30fc?<\/p>\n<p>\u53c2\u8003\u6587\u732e<\/p>\n<ol>\n<li><a href=\"http:\/\/www.robotstxt.org\/wc\/norobots.html\">A Standard for Robot Exclusion<\/a>, <a href=\"http:\/\/www.robotstxt.org\/\">robotstxt.org<\/a><\/li>\n<li><a href=\"http:\/\/blog.browncat.org\/2007\/05\/baidunaverwiki.html\">\u5642\u306eBaidu\u3068Naver\u306e\u30af\u30ed\u30fc\u30e9\u306fwiki\u304c\u304a\u597d\u304d<\/a>, <a href=\"http:\/\/blog.browncat.org\/\">blog@browncat.org<\/a>, 2007<\/li>\n<\/ol>\n<p><\/p>\n","protected":false},"excerpt":{"rendered":"<p>\u5148\u65e5\u306e\u8a18\u4e8b\u3092\u66f8\u3044\u3066robots.txt\u3092\u8a2d\u5b9a\u3057\u305f\u3042\u3068\u3001\u3071\u3063\u305f\u308a\u3053\u306a\u304f\u306a\u3063\u3066\u3044\u305fyetibot@naver.com\u3055\u3093\u3067\u3059\u304c\u3001robots.txt\u306e\u5236\u9650\u3092\u5916\u3057\u3066\u304a\u3044\u305f\u304b\u3044\u3042\u3063\u3066(?)\u300125\u65e5\u306e\u671d9\u6642\u9803\u304b\u3089\u6d3b\u52d5\u3092\u518d\u958b\u3057\u305f\u3088\u3046\u3067\u3059\u3002\u4f8b\u306e\u30b5\u30a4\u30c8\u3078\u306e\u30a2\u30af\u30bb\u30b9\u304c25\u65e5\u306e\u671d9\u6642\u304b\u308926\u65e5\u306e\u671d3\u6642\u307e\u3067\u30674247\u30a2\u30af\u30bb\u30b9\u3067\u3057\u305f\u3002\u305d\u306e\u3046\u3061\u7d04\u534a\u6570\u306e2125\u30a2\u30af\u30bb\u30b9\u304ccss\u30d5\u30a1\u30a4\u30eb\u3067\u3059\u3002\u4f8b\u306b\u3088\u3063\u3066\u3002 \u4eca\u56de\u306f\u3044\u308d\u3044\u308d\u5b9f\u9a13\u3057\u3066\u307f\u305f\u3044\u306e\u3067\u3001\u7d20\u76f4\u306b\u306f\u3058\u304f\u306e\u306f\u3084\u3081\u3066\u3001robots.txt\u306b\u6b21\u306e\u3088\u3046\u306b\u66f8\u3044\u3066\u307f\u307e\u3057\u305f\u3002 User-agent: Yeti Disallow: \/css\/ Disallow: \/img\/ Crawl-delay: 180 User-agent: Baidu Disallow: \/css\/ Disallow: \/img\/ Crawl-delay: 180 User-agent: * Disallow: Crawl-delay\u306fA Standard for Robot Exclusion\u306b\u306f\u542b\u307e\u308c\u3066\u3044\u306a\u3044\u3088\u3046\u3067\u3059\u304c\u3001ask.com\u3084Yahoo!\u306e\u30dc\u30c3\u30c8\u306f\u30b5\u30dd\u30fc\u30c8\u3057\u3066\u3044\u308b\u3088\u3046\u3067\u3059\u3002\u53c2\u8003\u6587\u732e[2]\u306b\u3088\u308b\u3068Yeti\u3055\u3093\u306fCrawl-delay\u3092\u7406\u89e3\u3067\u304d\u306a\u3044\u3088\u3046\u3067\u3059\u304c\u3001\u307e\u3042\u3044\u3061\u304a\u3046\u308f\u305f\u3057\u3082\u8a66\u3057\u3066\u307f\u305f\u3044\u306e\u3067\u3001\u66f8\u3044\u3066\u307f\u307e\u3057\u305f\u3002\u3042\u3068\u306fcss\u3092\u6bce\u56de(!)\u8aad\u307f\u306b\u304d\u3066\u3046\u308b\u3055\u3044\u306e\u3067\u3001\u3053\u308c\u3082\u7981\u6b62\u306b\u3002\u30a4\u30e1\u30fc\u30b8\u30d5\u30a1\u30a4\u30eb\u306f\u8aad\u307f\u306b\u304d\u3066\u306f\u3044\u306a\u3044\u3051\u3069\u3001\u5ff5\u306e\u305f\u3081\u3002 \u3064\u3044\u3067\u306b(?)Baidu\u3055\u3093\u3082\u6307\u5b9a\u3057\u3066\u307f\u305f\u3002\u73fe\u5728\u306e\u3068\u3053\u308d\u3001Baiduspider\u3055\u3093\u306f\u30b3\u30f3\u30b9\u30bf\u30f3\u30c8\u306b2\u5206\u306b1\u30a2\u30af\u30bb\u30b9\u30011\u6642\u9593\u306b30\u56dex24\u6642\u9593=720\u30a2\u30af\u30bb\u30b9\u7a0b\u5ea6\u306e\u300c\u826f\u8b58\u7684\u300d\u30a2\u30af\u30bb\u30b9\u3067\u3059\u3002\u3068\u308a\u3042\u3048\u305a\u9593\u9694\u30923\u5206\u306b\u3057\u3066\u3082\u3089\u3048\u308b\u304b\u3069\u3046\u304b\u30c6\u30b9\u30c8\u306e\u305f\u3081\u52a0\u3048\u3066\u307f\u307e\u3057\u305f\u3002 \u3055\u3042\u3069\u3046\u306a\u308b\u304b\u306a\u30fc? \u53c2\u8003\u6587\u732e A Standard for Robot Exclusion, robotstxt.org \u5642\u306eBaidu\u3068Naver\u306e\u30af\u30ed\u30fc\u30e9\u306fwiki\u304c\u304a\u597d\u304d, blog@browncat.org, 2007<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[43,34],"tags":[],"class_list":["post-443","post","type-post","status-publish","format-standard","hentry","category-web","category-34"],"_links":{"self":[{"href":"https:\/\/iwatakenichi.com\/index.php?rest_route=\/wp\/v2\/posts\/443","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/iwatakenichi.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/iwatakenichi.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/iwatakenichi.com\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/iwatakenichi.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=443"}],"version-history":[{"count":0,"href":"https:\/\/iwatakenichi.com\/index.php?rest_route=\/wp\/v2\/posts\/443\/revisions"}],"wp:attachment":[{"href":"https:\/\/iwatakenichi.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=443"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/iwatakenichi.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=443"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/iwatakenichi.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=443"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}