{"id":5227,"date":"2021-03-16T21:24:45","date_gmt":"2021-03-16T18:24:45","guid":{"rendered":"https:\/\/www.niobehosting.com\/blog\/?p=5227"},"modified":"2021-05-03T11:47:11","modified_gmt":"2021-05-03T08:47:11","slug":"robot-txt","status":"publish","type":"post","link":"https:\/\/www.niobehosting.com\/blog\/robot-txt\/","title":{"rendered":"Robot.txt Nedir? Robots TXT Dosyas\u0131 Nas\u0131l Olu\u015fturulur?"},"content":{"rendered":"\n<p><strong>Robots.txt dosyas\u0131<\/strong>, bir sitenin k\u00f6k dizininde bulunan ve arama motoru taray\u0131c\u0131lar\u0131na y\u00f6nelik olarak hangi URL&#8217;lerin taranmas\u0131 ve hangilerinin taranmamas\u0131 gerekti\u011fi hakk\u0131nda bilgiler i\u00e7eren bir metin belgesidir. Bu dosyan\u0131n varl\u0131\u011f\u0131 web sitesinin \u00e7al\u0131\u015fmas\u0131 i\u00e7in zorunlu de\u011fildir, ancak SEO i\u00e7in \u00e7ok \u00f6nemlidir.<\/p>\n\n\n\n<p>Robots.txt kullanma karar\u0131, Robot Exclusion Standard\u2019\u0131n bir par\u00e7as\u0131 olarak 1994 y\u0131l\u0131nda kabul edildi.<a href=\"https:\/\/developers.google.com\/search\/docs\/advanced\/robots\/intro?hl=en&amp;visit_id=637485091797062512-2328977254&amp;rd=1\" target=\"_blank\" rel=\"noreferrer noopener nofollow\"> Google Yard\u0131m Merkezi<\/a>&#8216;ne g\u00f6re dosyan\u0131n temel amac\u0131, web sayfalar\u0131n\u0131n arama sonu\u00e7lar\u0131nda g\u00f6sterilmesini engellemek de\u011fil, robotlar\u0131n sitelere yapt\u0131\u011f\u0131 istek say\u0131s\u0131n\u0131 s\u0131n\u0131rlamak ve sunucu y\u00fck\u00fcn\u00fc azaltmakt\u0131r.<\/p>\n\n\n\n<p>Genel olarak robots.txt dosyas\u0131n\u0131n i\u00e7eri\u011fi, web sitesi taramas\u0131na ili\u015fkin kurallar\u0131 tan\u0131mlayan arama taray\u0131c\u0131lar\u0131 i\u00e7in bir \u00f6neri olarak g\u00f6r\u00fclmelidir. Herhangi bir sitenin robots.txt dosyas\u0131n\u0131n i\u00e7eri\u011fine eri\u015fmek i\u00e7in yapman\u0131z gereken tek \u015fey, taray\u0131c\u0131daki alan ad\u0131ndan sonra \u201c\/robots.txt\u201d yazmakt\u0131r.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Robots.txt Ne \u0130\u00e7in Kullan\u0131l\u0131r?<\/h2>\n\n\n\n<p><strong>Robots.txt <\/strong>dosyas\u0131n\u0131n birincil amac\u0131, tarama b\u00fct\u00e7esinin (bir arama robotunun tarayabilece\u011fi web sayfalar\u0131n\u0131n say\u0131s\u0131d\u0131r) daha verimli bir \u015fekilde tahsis edilmesi i\u00e7in sayfalar\u0131n ve kaynak dosyalar\u0131n\u0131n taranmas\u0131n\u0131 \u00f6nlemektir. \u00c7o\u011fu durumda robots.txt dosyas\u0131, web sitesi ziyaret\u00e7ilerine herhangi bir de\u011fer sa\u011flamayan ve SERP s\u0131ralamas\u0131n\u0131 etkilemeyen bilgileri gizler.<\/p>\n\n\n\n<p><strong>Robots.txt avantajlar\u0131n\u0131 <\/strong>daha detayl\u0131 anlatmak gerekirse:<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">1. Botlar\u0131 \u00d6zel Klas\u00f6rlerden Uzak Tutma<\/h3>\n\n\n\n<p>Botlar\u0131n \u00f6zel klas\u00f6rlerinizi kontrol etmesini \u00f6nlemek, onlar\u0131 bulmay\u0131 ve indekslemeyi \u00e7ok daha zor hale getirecektir.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">2. Kaynaklar\u0131 Kontrol Alt\u0131nda Tutma<\/h3>\n\n\n\n<p>Bir bot sitenizde her gezindi\u011finde, bant geni\u015fli\u011fini ve di\u011fer sunucu kaynaklar\u0131n\u0131 emer. \u00d6rne\u011fin, e-ticaret siteleri gibi tonlarca i\u00e7eri\u011fe ve \u00e7ok say\u0131da sayfaya sahip siteler i\u00e7in, bu kaynaklar ger\u00e7ekten h\u0131zl\u0131 bir \u015fekilde t\u00fcketilebilir. Robotlar\u0131n ba\u011f\u0131ms\u0131z komut dosyalar\u0131na ve g\u00f6r\u00fcnt\u00fclere eri\u015fmesini zorla\u015ft\u0131rmak i\u00e7in robots.txt dosyas\u0131n\u0131 kullanabilirsiniz; bu, ger\u00e7ek ziyaret\u00e7iler i\u00e7in de\u011ferli kaynaklar\u0131 koruyacakt\u0131r.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">3. Site Haritan\u0131z\u0131n Konumunu Belirtme<\/h3>\n\n\n\n<p>Bu olduk\u00e7a \u00f6nemli bir noktad\u0131r. Taray\u0131c\u0131lara site haritan\u0131z\u0131n nerede oldu\u011funu bildirmek istersiniz, b\u00f6ylece site haritan\u0131z\u0131 tarayabilirler.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">4. Yinelenen \u0130\u00e7eri\u011fi SERP&#8217;lerden Uzak Tutma<\/h3>\n\n\n\n<p>Kural\u0131 robotlar\u0131n\u0131za ekleyerek, taray\u0131c\u0131lar\u0131n yinelenen i\u00e7eri\u011fin bulundu\u011fu sayfalar\u0131 endekslemesini \u00f6nleyebilirsiniz.<\/p>\n\n\n\n<p>Do\u011fal olarak, arama motorlar\u0131n\u0131n web sitenizdeki en \u00f6nemli sayfalara giden yolu bulmas\u0131n\u0131 isteyeceksiniz. Belirli sayfalar\u0131 belirli kurallarla kontrol alt\u0131na alarak, hangi sayfalar\u0131n arama yapanlar\u0131n \u00f6n\u00fcne konulaca\u011f\u0131n\u0131 kontrol edebilirsiniz.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Robots.txt Arac\u0131l\u0131\u011f\u0131yla Genellikle Hangi Sayfalar ve Dizinler Engellenebilir?<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">1. Ki\u015fisel veriler i\u00e7eren sayfalar<\/h3>\n\n\n\n<p>Ki\u015fisel veriler, ziyaret\u00e7ilerin kay\u0131t s\u0131ras\u0131nda belirttikleri isimleri ve telefon numaralar\u0131n\u0131, ki\u015fisel g\u00f6sterge tablolar\u0131n\u0131 ve profil sayfalar\u0131n\u0131, \u00f6deme kart\u0131 numaralar\u0131n\u0131 i\u00e7erebilir. G\u00fcvenlik nedeniyle, bu t\u00fcr bilgilere eri\u015fim ek olarak bir parola ile korunmal\u0131d\u0131r.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">2. Yaln\u0131zca belirli kullan\u0131c\u0131 eylemlerinden sonra g\u00f6r\u00fcnen yard\u0131mc\u0131 sayfalar<\/h3>\n\n\n\n<p>Bu t\u00fcr eylemler tipik olarak m\u00fc\u015fterilerin bir sipari\u015fi, m\u00fc\u015fteri formlar\u0131n\u0131, yetkilendirme veya \u015fifre kurtarma sayfalar\u0131n\u0131 ba\u015far\u0131yla tamamlad\u0131ktan sonra ald\u0131klar\u0131 mesajlar\u0131 i\u00e7erir.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">3. Y\u00f6netici kontrol paneli ve sistem dosyalar\u0131<\/h3>\n\n\n\n<p>Web sitesi y\u00f6neticilerinin veya web y\u00f6neticisinin etkile\u015fimde bulundu\u011fu dahili dosyalar ve hizmet dosyalar\u0131n\u0131 i\u00e7erir.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">4. Arama ve kategori s\u0131ralama sayfalar\u0131<\/h3>\n\n\n\n<p>Bir web sitesi ziyaret\u00e7isi sitenin arama kutusuna bir sorgu girdikten sonra g\u00f6r\u00fcnt\u00fclenen sayfalar genellikle arama motoru taray\u0131c\u0131lar\u0131na kapat\u0131l\u0131r. Ayn\u0131 \u015fey, kullan\u0131c\u0131lar\u0131n \u00fcr\u00fcnleri fiyat, derecelendirme ve di\u011fer kriterlere g\u00f6re s\u0131ralarken elde etti\u011fi sonu\u00e7lar i\u00e7in de ge\u00e7erlidir.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">5. Belirli bir formattaki dosyalar<\/h3>\n\n\n\n<p>Bu t\u00fcr dosyalar foto\u011fraflar\u0131, videolar\u0131, .PDF belgelerini, JS dosyalar\u0131n\u0131 i\u00e7erebilir. Robots.txt yard\u0131m\u0131yla, tek tek veya uzant\u0131ya \u00f6zg\u00fc dosyalar\u0131n taranmas\u0131n\u0131 k\u0131s\u0131tlayabilirsiniz.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Robots.txt Dosyas\u0131 Nas\u0131l Olu\u015fturulur ve Nereye Yerle\u015ftirilir?<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">1. Robots.txt kurulumu i\u00e7in ara\u00e7lar<\/h3>\n\n\n\n<p>Belgenin bir .txt uzant\u0131s\u0131 oldu\u011fundan, <strong>UTF-8<\/strong> kodlamas\u0131n\u0131 destekleyen herhangi bir metin d\u00fczenleyici uygun olacakt\u0131r. En kolay se\u00e7enek Notepad (Windows) veya TextEdit\u2019tir (Mac).<\/p>\n\n\n\n<p>Ayr\u0131ca, belirtilen bilgilere dayanarak bir robots.txt dosyas\u0131 olu\u015fturacak bir <strong>robots.txt<\/strong> olu\u015fturma arac\u0131 da kullanabilirsiniz.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">2. Belge ba\u015fl\u0131\u011f\u0131 ve boyutu<\/h3>\n\n\n\n<p>robots.txt dosyas\u0131n\u0131n ad\u0131, b\u00fcy\u00fck harf kullan\u0131lmadan tam olarak b\u00f6yle g\u00f6r\u00fcnmelidir.<a href=\"https:\/\/developers.google.com\/search\/reference\/robots_txt\" target=\"_blank\" rel=\"noreferrer noopener\"> Google<\/a>\u2019ye g\u00f6re, izin verilen belge boyutu 500 KB&#8217;dir. Bu s\u0131n\u0131r\u0131n a\u015f\u0131lmas\u0131, arama robotunun belgeyi k\u0131smen i\u015flemesine, web sitesini hi\u00e7 taramamas\u0131na veya tersine, bir web sitesinin i\u00e7eri\u011fini tamamen taramas\u0131na neden olabilir.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">3. Dosya nereye yerle\u015ftirilir?<\/h3>\n\n\n\n<p>Belge, web sitesi ana bilgisayar\u0131n\u0131n <strong>k\u00f6k dizininde<\/strong> bulunmal\u0131d\u0131r (<a href=\"https:\/\/www.niobehosting.com\/blog\/ftp\/\">FTP<\/a> yoluyla eri\u015filebilir). Herhangi bir de\u011fi\u015fiklik yapmadan \u00f6nce, robots.txt dosyas\u0131n\u0131 orijinal bi\u00e7iminde indirmeniz \u00f6nerilir.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Robots.txt Dosyan\u0131za Neler Koymal\u0131s\u0131n\u0131z?<\/h2>\n\n\n\n<p>Tamam, art\u0131k sunucunuzda gerekti\u011finde d\u00fczenleyebilece\u011finiz fiziksel bir <strong>robots.txt<\/strong> dosyan\u0131z var. Ama o dosyayla ger\u00e7ekte ne yapars\u0131n\u0131z? \u0130lk b\u00f6l\u00fcmde \u00f6\u011frendi\u011finiz gibi robots.txt, robotlar\u0131n sitenizle nas\u0131l etkile\u015fim kurdu\u011funu kontrol etmenizi sa\u011flar. Bunu iki temel komutla yapars\u0131n\u0131z:<\/p>\n\n\n\n<p><strong>User-agent:<\/strong> Bu, belirli botlar\u0131 hedeflemenizi sa\u011flar. User-agent, botlar\u0131n kendilerini tan\u0131mlamak i\u00e7in kulland\u0131klar\u0131 \u015feydir. Bunlarla, \u00f6rne\u011fin Bing i\u00e7in ge\u00e7erli olan ancak Google i\u00e7in ge\u00e7erli olmayan bir kural olu\u015fturabilirsiniz.<\/p>\n\n\n\n<p><strong>Disallow:<\/strong> Bu, robotlara sitenizin belirli alanlar\u0131na eri\u015fmemelerini s\u00f6ylemenizi sa\u011flar.<\/p>\n\n\n\n<p>Ni\u015f durumlarda kullanaca\u011f\u0131n\u0131z bir<strong> Allow<\/strong> komutu da vard\u0131r. Varsay\u0131lan olarak, sitenizdeki her \u015fey Allow olarak i\u015faretlenmi\u015ftir, bu nedenle genellikle Allow komutunun kullan\u0131lmas\u0131 gerekli de\u011fildir. Ancak, bir klas\u00f6re ve alt klas\u00f6rlerine eri\u015fime izin vermemek, ancak belirli bir alt klas\u00f6re eri\u015fime izin vermek istedi\u011finizde i\u015fe yarar.<\/p>\n\n\n\n<p>\u0130lk olarak kural\u0131n hangi User-agent uygulanmas\u0131 gerekti\u011fini belirleyip ard\u0131ndan Disallow ve Allow se\u00e7eneklerini kullanarak hangi kurallar\u0131n uygulanaca\u011f\u0131n\u0131 listeleyerek kurallar eklersiniz. Crawl-delay ve Site Haritas\u0131 gibi ba\u015fka komutlar da vard\u0131r, ancak bunlar:<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>\u00c7o\u011fu taray\u0131c\u0131 taraf\u0131ndan yok say\u0131l\u0131r veya \u00e7ok farkl\u0131 \u015fekillerde yorumlan\u0131r (tarama gecikmesi durumunda)<\/li><li>Google Arama Konsolu gibi ara\u00e7lar taraf\u0131ndan gereksiz hale getirildi (site haritalar\u0131 i\u00e7in)<\/li><\/ul>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>Yararl\u0131 robots.txt Kurallar\u0131<\/strong><\/h2>\n\n\n\n<figure class=\"wp-block-table\"><table><tbody><tr><td><strong>Kural<\/strong><\/td><td>&nbsp;<\/td><\/tr><tr><td>Web sitesinin taranmas\u0131n\u0131 t\u00fcm\u00fcyle reddetme. Baz\u0131 durumlarda web sitesindeki URL&#8217;lerin, taranmam\u0131\u015f olsalar bile yine de dizine eklenebileceklerini unutmay\u0131n.Not: Bu kural, baz\u0131 AdsBot taray\u0131c\u0131lar\u0131yla e\u015fle\u015fmez. Bu taray\u0131c\u0131lar\u0131n adlar\u0131n\u0131n a\u00e7\u0131k\u00e7a belirtilmesi gerekir.<\/td><td>User-agent: *Disallow: \/<\/td><\/tr><tr><td>Dizin ad\u0131ndan sonra d\u00fcz e\u011fik \u00e7izgi ekleyerek bir dizin ve i\u00e7eri\u011fin taranmas\u0131na izin vermeme. Gizli i\u00e7eri\u011fe eri\u015fimi engellemek i\u00e7in robots.txt dosyas\u0131n\u0131 kullanmaman\u0131z gerekti\u011fini unutmay\u0131n. Bunun yerine uygun kimlik do\u011frulama y\u00f6ntemini kullan\u0131n. Robots.txt dosyas\u0131yla taranmas\u0131na izin verilmeyen URL&#8217;ler, taranmadan dizine eklenebilir ve robots.txt dosyas\u0131 ba\u015fkalar\u0131 taraf\u0131ndan g\u00f6r\u00fcnt\u00fclenerek gizli i\u00e7eri\u011finizin yerinin ortaya \u00e7\u0131kmas\u0131na neden olabilir.<\/td><td>User-agent: *Disallow: \/yonetim\/Disallow: \/izin\/<\/td><\/tr><tr><td>Tek bir taray\u0131c\u0131n\u0131n eri\u015fimine izin verme<\/td><td>User-agent: Googlebot-newsAllow: \/<br>User-agent: *Disallow: \/<\/td><\/tr><tr><td>Bir taray\u0131c\u0131 d\u0131\u015f\u0131nda t\u00fcm taray\u0131c\u0131lar\u0131n eri\u015fimine izin verme<\/td><td>User-agent: UnnecessarybotDisallow: \/<br>User-agent: *Allow: \/<\/td><\/tr><tr><td>E\u011fik \u00e7izgiden sonra sayfay\u0131 listeleyerek tek bir web sayfas\u0131n\u0131n taranmas\u0131na izin vermeme.<\/td><td>User-agent: *Disallow: \/ozel-dosya.html<\/td><\/tr><tr><td>Google G\u00f6rseller&#8217;den belirli bir g\u00f6rseli engelleme:<\/td><td>User-agent: Googlebot-ImageDisallow: \/gorsel\/urun-12.jpg<\/td><\/tr><tr><td>Google G\u00f6rseller&#8217;den sitenizdeki t\u00fcm g\u00f6rselleri engelleme<\/td><td>User-agent: Googlebot-ImageDisallow: \/<\/td><\/tr><tr><td>Belirli t\u00fcrdeki dosyalar\u0131n taranmas\u0131na izin vermeme (\u00f6rne\u011fin, .gif):<\/td><td>User-agent: GooglebotDisallow: \/*.gif$<\/td><\/tr><tr><td>Sitenizin tamam\u0131n\u0131n taranmas\u0131na izin vermeme; ancak bu sayfalarda AdSense reklamlar\u0131 g\u00f6sterme ve Mediapartners-Google d\u0131\u015f\u0131ndaki hi\u00e7bir web taray\u0131c\u0131s\u0131na izin vermeme. Bu uygulama, sayfalar\u0131n\u0131z\u0131 arama sonu\u00e7lar\u0131ndan gizler. Ancak Mediapartners-Google web taray\u0131c\u0131s\u0131, sitenizin ziyaret\u00e7ilerine hangi reklamlar\u0131n g\u00f6r\u00fcnt\u00fclenece\u011fine karar vermek \u00fczere sayfalar\u0131 analiz etmeye devam eder.<\/td><td>User-agent: *Disallow: \/<br>User-agent: Mediapartners-GoogleAllow: \/<\/td><\/tr><tr><td>Belirli bir dize ile biten URL&#8217;leri e\u015fle\u015ftirme; $ kullan\u0131n. \u00d6rne\u011fin, bu \u00f6rnek kod .xls ile biten t\u00fcm URL&#8217;leri engeller:<\/td><td>User-agent: GooglebotDisallow: \/*.xls$<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\">T\u00fcm Sitenize Eri\u015fimi Engellemek \u0130\u00e7in Robots.txt Nas\u0131l Kullan\u0131l\u0131r?<\/h2>\n\n\n\n<p>Taray\u0131c\u0131n\u0131n sitenize t\u00fcm eri\u015fimini engellemek istedi\u011finizi varsayal\u0131m. Bunun canl\u0131 bir sitede meydana gelmesi olas\u0131 de\u011fildir, ancak bir geli\u015ftirme sitesi i\u00e7in kullan\u0131\u015fl\u0131 olur. Bunu yapmak i\u00e7in, a\u015fa\u011f\u0131daki kodu WordPress robots.txt dosyan\u0131za ekleyin:<\/p>\n\n\n\n<p>User-agent: *<br>Disallow: \/<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Tek Bir Botun Sitenize Eri\u015fmesini Engellemek \u0130\u00e7in Robots.txt Nas\u0131l Kullan\u0131l\u0131r?<\/h2>\n\n\n\n<p>\u00d6rne\u011fin Bing&#8217;in sitenize bakmas\u0131n\u0131 bile istemiyorsunuz. Yaln\u0131zca Bing&#8217;in sitenizi taramas\u0131n\u0131 engellemek i\u00e7in a\u015fa\u011f\u0131daki kodu WordPress robots.txt dosyan\u0131za ekleyin:<\/p>\n\n\n\n<p>User-agent: Bingbot<br>Disallow: \/<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Belirli Bir Klas\u00f6re veya Dosyaya Eri\u015fimi Engellemek \u0130\u00e7in Robots.txt Nas\u0131l Kullan\u0131l\u0131r?<\/h2>\n\n\n\n<p>Yaln\u0131zca belirli bir dosya veya klas\u00f6re (ve bu klas\u00f6r\u00fcn t\u00fcm alt klas\u00f6rlerine) eri\u015fimi engellemek istedi\u011finizi varsayal\u0131m. Bunu WordPress i\u00e7in ge\u00e7erli k\u0131lmak i\u00e7in, \u015funu engellemek istedi\u011finizi varsayal\u0131m:<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>Wp-admin klas\u00f6r\u00fcn\u00fcn tamam\u0131<\/li><li>wp-login.php<\/li><\/ul>\n\n\n\n<p>A\u015fa\u011f\u0131daki komutlar\u0131 kullanabilirsiniz:<\/p>\n\n\n\n<p>User-agent: *<br>Disallow: \/wp-admin\/<br>Disallow: \/wp-login.php<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">\u0130zin Verilmeyen Bir Klas\u00f6rdeki Belirli Bir Dosyaya Eri\u015fime \u0130zin Vermek \u0130\u00e7in Robots.txt Nas\u0131l Kullan\u0131l\u0131r?<\/h2>\n\n\n\n<p>Diyelim ki t\u00fcm bir klas\u00f6r\u00fc engellemek istiyorsunuz, ancak yine de o klas\u00f6r i\u00e7indeki belirli bir dosyaya eri\u015fime izin vermek istiyorsunuz. \u0130zin ver komutunun kullan\u0131\u015fl\u0131 oldu\u011fu yer buras\u0131d\u0131r.&nbsp;<\/p>\n\n\n\n<p>User-agent: *<br>Disallow: \/wp-admin\/<br>Allow: \/wp-admin\/admin-ajax.php<\/p>\n\n\n\n<p>Bu kod par\u00e7as\u0131, \/wp-admin\/admin-ajax.php dosyas\u0131 d\u0131\u015f\u0131nda \/ wp-admin \/ klas\u00f6r\u00fcn\u00fcn tamam\u0131na eri\u015fimi engeller.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Robotlar\u0131n WordPress Arama Sonu\u00e7lar\u0131n\u0131 Taramas\u0131n\u0131 Durdurmak \u0130\u00e7in Robots.txt Nas\u0131l Kullan\u0131l\u0131r?<\/h2>\n\n\n\n<p>Bu a\u015famada yapman\u0131z gereken \u015fey, arama taray\u0131c\u0131lar\u0131n\u0131n arama sonu\u00e7lar\u0131 sayfalar\u0131n\u0131z\u0131 taramas\u0131n\u0131 durdurmakt\u0131r. WordPress varsay\u0131lan olarak,&#8221;?s=&#8221; sorgu parametresini kullan\u0131r. Yani eri\u015fimi engellemek i\u00e7in tek yapman\u0131z gereken a\u015fa\u011f\u0131daki kural\u0131 eklemektir:<\/p>\n\n\n\n<p>User-agent: *<br>Disallow: \/?s=<br>Disallow: \/search\/<\/p>\n\n\n\n<p>Bu,&nbsp; 404 hatalar\u0131n\u0131 ald\u0131\u011f\u0131n\u0131zda da durdurman\u0131n etkili bir yolu olabilir.&nbsp;<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Robots.txt&#8217;de Farkl\u0131 Botlar \u0130\u00e7in Farkl\u0131 Kurallar Nas\u0131l Olu\u015fturulur?<\/h2>\n\n\n\n<p>\u015eimdiye kadar, t\u00fcm \u00f6rnekler bir seferde tek bir kuralla ilgiliydi. Peki ya farkl\u0131 botlara farkl\u0131 kurallar uygulamak isterseniz? Her bot i\u00e7in User-agent beyan\u0131n\u0131n alt\u0131na her bir kural k\u00fcmesini eklemeniz yeterlidir. \u00d6rne\u011fin, t\u00fcm botlar i\u00e7in ge\u00e7erli olan bir kural ve yaln\u0131zca Bingbot i\u00e7in ge\u00e7erli olan ba\u015fka bir kural olu\u015fturmak istiyorsan\u0131z, bunu a\u015fa\u011f\u0131daki \u015fekilde yapabilirsiniz:<\/p>\n\n\n\n<p>User-agent: *<br>Disallow: \/wp-admin\/<br>User-agent: Bingbot<br>Disallow: \/<\/p>\n\n\n\n<p>Bu \u00f6rnekte, t\u00fcm botlar\u0131n \/ wp-admin \/ eri\u015fimi engellenecek, ancak Bingbot&#8217;un t\u00fcm sitenize eri\u015fimi engellenecektir.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>Ge\u00e7erli robots.txt URL&#8217;leri \u0130\u00e7in \u00d6rnekler<\/strong><\/h2>\n\n\n\n<figure class=\"wp-block-table\"><table><tbody><tr><td><strong>Robots.txt URL \u00f6rnekleri<\/strong><\/td><\/tr><tr><td>http:\/\/niobe.com\/robots.txt<\/td><td><strong>Ge\u00e7erli:<\/strong><br>http:\/\/niobe.com\/<br>http:\/\/niobe.com\/klasor\/dosya<br><strong>Ge\u00e7erli de\u011fil:<\/strong><br>http:\/\/blog.niobe.com\/<br>https:\/\/niobe.com\/<br>http:\/\/niobe.com:8181\/<br>Bu, genel kullan\u0131md\u0131r. Di\u011fer alt alan adlar\u0131, protokoller veya ba\u011flant\u0131 noktas\u0131 numaralar\u0131 i\u00e7in ge\u00e7erli de\u011fildir. Ayn\u0131 ana makine, protokol ve ba\u011flant\u0131 noktas\u0131 numaras\u0131nda bulunan t\u00fcm alt dizinlerdeki dosyalar\u0131n tamam\u0131 i\u00e7in ge\u00e7erlidir.<\/td><\/tr><tr><td>http:\/\/www.niobe.com\/robots.txt<\/td><td><strong>Ge\u00e7erli:<\/strong> <br>http:\/\/www.niobe.com\/<br><strong>Ge\u00e7erli de\u011fil:<\/strong><br>http:\/\/niobe.com\/<br>http:\/\/blog.www.niobe.com\/<br>http:\/\/www.blog.niobe.com\/<br>Bir alt alan ad\u0131ndaki robots.txt dosyas\u0131 yaln\u0131zca s\u00f6z konusu alt alan ad\u0131 i\u00e7in ge\u00e7erlidir.<\/td><\/tr><tr><td>http:\/\/niobe.com\/folder\/robots.txt<\/td><td>Ge\u00e7erli bir robots.txt dosyas\u0131 de\u011fildir! Taray\u0131c\u0131lar, alt dizinlerdeki robots.txt dosyalar\u0131n\u0131 kontrol etmez.<\/td><\/tr><tr><td>http:\/\/www.m\u00fcller.eu\/robots.txt<\/td><td><strong>Ge\u00e7erli:<\/strong><br>http:\/\/www.m\u00fcller.eu\/<br>http:\/\/www.xn--mller-kva.eu\/<br><strong>Ge\u00e7erli de\u011fil:<\/strong><br>http:\/\/www.muller.eu\/IDN&#8217;ler (Uluslararas\u0131 Hale Getirilmi\u015f Alan Ad\u0131), punycode versiyonlar\u0131n e\u015f de\u011feridir. <\/td><\/tr><tr><td>ftp:\/\/niobe.com\/robots.txt<\/td><td><strong>Ge\u00e7erli:<\/strong> <br>ftp:\/\/niobe.com\/<br><strong>Ge\u00e7erli de\u011fil:<\/strong><br>http:\/\/niobe.com\/Google&#8217;a \u00f6zel: robots.txt dosyas\u0131n\u0131 FTP kaynaklar\u0131 i\u00e7in de kullan\u0131r\u0131z.<\/td><\/tr><tr><td>http:\/\/212.96.82.21\/robots.txt<\/td><td><strong>Ge\u00e7erli:<\/strong> <br>http:\/\/212.96.82.21\/<br><strong>Ge\u00e7erli de\u011fil:<\/strong> <br>http:\/\/niobe.com\/ (212.96.82.21 adresinde bar\u0131nd\u0131r\u0131lsa bile)Ana makine ad\u0131yla ayn\u0131 IP adresine sahip bir robots.txt dosyas\u0131, ana makine ad\u0131 olarak yaln\u0131zca s\u00f6z konusu IP adresinin taranmas\u0131 i\u00e7in ge\u00e7erli olur. S\u00f6z konusu IP adresinde bar\u0131nd\u0131r\u0131lan t\u00fcm web siteleri i\u00e7in otomatik olarak ge\u00e7erli olmaz (bununla birlikte, robots.txt dosyas\u0131n\u0131n payla\u015f\u0131lmas\u0131 m\u00fcmk\u00fcnd\u00fcr. Bu durumda, robots.txt dosyas\u0131, payla\u015f\u0131lan ana makine ad\u0131 alt\u0131nda da kullan\u0131labilir).<\/td><\/tr><tr><td>http:\/\/niobe.com:80\/robots.txt<\/td><td><strong>Ge\u00e7erli:<\/strong><br>http:\/\/niobe.com:80\/<br>http:\/\/niobe.com\/<br><strong>Ge\u00e7erli de\u011fil:<\/strong><br>http:\/\/niobe.com:81\/<br>Standart ba\u011flant\u0131 noktas\u0131 numaralar\u0131, (http i\u00e7in 80, https i\u00e7in 443, ftp i\u00e7in 21), varsay\u0131lan ana makine adlar\u0131na e\u015f de\u011ferdir. Ayr\u0131ca bkz. <strong>[portnumbers]<\/strong>.<\/td><\/tr><tr><td>http:\/\/niobe.com:8181\/robots.txt<\/td><td><strong>Ge\u00e7erli:<\/strong> <br>http:\/\/niobe.com:8181\/<br><strong>Ge\u00e7erli de\u011fil:<\/strong><br>http:\/\/niobe.com\/<br>Standart olmayan ba\u011flant\u0131 noktas\u0131 numaralar\u0131ndaki robots.txt dosyalar\u0131, yaln\u0131zca bu ba\u011flant\u0131 noktas\u0131 numaralar\u0131 \u00fczerinden kullan\u0131ma sunulan i\u00e7erik i\u00e7in ge\u00e7erlidir.<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\">Robots.txt Dosyas\u0131n\u0131 Test Etme<\/h2>\n\n\n\n<p>Robots.txt dosyan\u0131z\u0131 bekledi\u011finiz gibi \u00e7al\u0131\u015ft\u0131\u011f\u0131ndan emin olmak i\u00e7in test edebilirsiniz. T\u00fcm\u00fcn\u00fcn do\u011fru oldu\u011funu d\u00fc\u015f\u00fcnseniz bile bunu yapman\u0131zda fayda var.<\/p>\n\n\n\n<p>Google Search Console\u2019ye (Google Arama Konsolu) girin (kurulumdan sonra) ve sol taraftaki Kapsam raporuna (Coverage report) gidin. Rapor &#8220;Hari\u00e7 tutulan (Excluded)&#8221; sekmesinde web sitesinde mevcut uyar\u0131lar\u0131, hatalar\u0131 ve engellenen sayfalarla ilgili di\u011fer bilgileri size g\u00f6sterir.&nbsp;<\/p>\n\n\n\n<p>Ya da Robots.txt dosyan\u0131z\u0131 <strong><a href=\"https:\/\/support.google.com\/webmasters\/answer\/6062598?hl=tr\" target=\"_blank\" rel=\"noreferrer noopener nofollow\">robots.txt Test Arac\u0131<\/a><\/strong> ile test edebilirsiniz.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Robots.txt Dosyan\u0131z\u0131 Test Etme<\/h2>\n\n\n\n<ol class=\"wp-block-list\"><li>Test arac\u0131n\u0131 a\u00e7t\u0131ktan sonra vurgulanan s\u00f6z dizimi uyar\u0131lar\u0131n\u0131 ve mant\u0131k hatalar\u0131n\u0131 bulmak i\u00e7in robots.txt kodunu kayd\u0131r\u0131n. S\u00f6zdizimi uyar\u0131lar\u0131n\u0131n ve mant\u0131k hatalar\u0131n\u0131n say\u0131s\u0131, d\u00fczenleyicinin hemen alt\u0131nda g\u00f6sterilir.<\/li><li>Sayfan\u0131n alt\u0131nda bulunan metin kutusuna web sitenizde bulunan bir sayfan\u0131n URL&#8217;sini yaz\u0131n.<\/li><li>Metin kutusunun sa\u011f\u0131ndaki a\u00e7\u0131l\u0131r listeden sim\u00fcle etmek istedi\u011finiz kullan\u0131c\u0131 arac\u0131s\u0131n\u0131 se\u00e7in.<\/li><li>Eri\u015fimi test etmek i\u00e7in \u201c<strong>Test<\/strong>\u201d d\u00fc\u011fmesine t\u0131klay\u0131n.<\/li><li>Girdi\u011finiz URL&#8217;nin Google web taray\u0131c\u0131lar\u0131 taraf\u0131ndan engellenip engellenmedi\u011fini \u00f6\u011frenmek i\u00e7in \u201c<strong>Test<\/strong>\u201d d\u00fc\u011fmesinin kontrol\u00fcn\u00fc sa\u011flayarak \u201c<strong>Kabul Edildi<\/strong>\u201d mi yoksa \u201c<strong>Engellendi<\/strong>\u201d mi yazd\u0131\u011f\u0131na bak\u0131n.<\/li><li>Sayfadaki dosyay\u0131 d\u00fczenleyip gerekirse tekrar test edin. Yap\u0131lan de\u011fi\u015fikliklerin sitenize kaydedilmedi\u011fini bilin ve bir sonraki ad\u0131ma ge\u00e7in.<\/li><li>Yap\u0131lan de\u011fi\u015fiklikleri sitenizde bulunan robots.txt dosyan\u0131za kopyalay\u0131n. Bu ara\u00e7 sitenizdeki ger\u00e7ek dosyada de\u011fi\u015fiklik yapmaz, testi sadece ara\u00e7ta bar\u0131nd\u0131r\u0131lan kopyay\u0131 kullanarak ger\u00e7ekle\u015ftirir.<\/li><\/ol>\n\n\n\n<h2 class=\"wp-block-heading\">Robots.txt Dosyan\u0131z Yoksa Ne Olur?<\/h2>\n\n\n\n<p>Bir robots.txt dosyas\u0131 olmadan arama motorlar\u0131, web sitesinde bulduklar\u0131 her \u015feyi taramak ve dizine eklemek i\u00e7in \u00fccretsiz bir \u00e7al\u0131\u015fma yapacakt\u0131r. Bu \u00e7o\u011fu web sitesi i\u00e7in uygundur, ancak en az\u0131ndan <a href=\"https:\/\/www.niobehosting.com\/blog\/site-haritasi\/\">XML site haritan\u0131z\u0131n<\/a> nerede oldu\u011funu belirtmeniz gerekir. B\u00f6ylece arama motorlar\u0131 web sitenizde yeni i\u00e7eri\u011fi h\u0131zla bulabilir ve tarama b\u00fct\u00e7elerini optimize edebilir.<\/p>\n\n\n\n<p>\u00d6zetlemek gerekirse:<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>Robots.txt dosyas\u0131, robotlara hangi sayfalar\u0131n taranmas\u0131 ve taranmamas\u0131 gerekti\u011fini s\u00f6yleyen bir k\u0131lavuzdur.<\/li><li>Robots.txt dosyas\u0131, endekslemeyi engelleyecek \u015fekilde yap\u0131land\u0131r\u0131lamaz, ancak bir robotun tarama veya belirli belgeleri veya dosyalar\u0131 yok sayma \u015fans\u0131n\u0131 art\u0131rabilirsiniz.<\/li><li>Yarars\u0131z web sitesi i\u00e7eri\u011fini disallow y\u00f6nergesiyle gizlemek, tarama b\u00fct\u00e7esinden tasarruf sa\u011flar. Bu hem \u00e7ok sayfal\u0131 hem de k\u00fc\u00e7\u00fck web siteleri i\u00e7in ge\u00e7erlidir.<\/li><li>Bir robots.txt dosyas\u0131 olu\u015fturmak i\u00e7in basit bir metin d\u00fczenleyici ve kontrol yapmak i\u00e7in Google Arama Konsolu yeterlidir.<\/li><li>Robots.txt dosyas\u0131n\u0131n ad\u0131 k\u00fc\u00e7\u00fck harflerle yaz\u0131lmal\u0131 ve 500 KB&#8217;yi ge\u00e7memelidir.<\/li><\/ul>\n\n\n\n<h2 class=\"wp-block-heading\">Robots.txt \u0130le E-ticaret Sitelerinin Tarama B\u00fct\u00e7esi Nas\u0131l Optimize Edilir?<\/h2>\n\n\n\n<p>Bir sayfan\u0131n arama sonu\u00e7lar\u0131nda g\u00f6r\u00fcnmesi ve sitenize trafik \u00e7ekmesi i\u00e7in, Google&#8217;nin \u00f6nce sayfay\u0131 taramas\u0131 gerekir. <a href=\"https:\/\/developers.google.com\/search\/blog\/2017\/01\/what-crawl-budget-means-for-googlebot\" target=\"_blank\" rel=\"noreferrer noopener nofollow\">Google&#8217;\u0131n kendi ifadesiyle<\/a>, <strong>&#8220;Tarama, sitelerin Google&#8217;nin arama sonu\u00e7lar\u0131na giri\u015f noktas\u0131d\u0131r.&#8221;<\/strong> Ancak, Google&#8217;nin web \u00fczerindeki her sayfay\u0131 her zaman taramak i\u00e7in s\u0131n\u0131rs\u0131z zaman\u0131 ve kayna\u011f\u0131 olmad\u0131\u011f\u0131 i\u00e7in t\u00fcm sayfalar taranmayacakt\u0131r.<\/p>\n\n\n\n<p>Bu, <a href=\"https:\/\/www.niobehosting.com\/blog\/seo\/\">SEO<\/a>&#8216;da <strong>tarama b\u00fct\u00e7esi<\/strong> olarak adland\u0131r\u0131lan kavramd\u0131r ve bunu optimize etmek, e-ticaret sitenizin b\u00fcy\u00fcmesinin anahtar\u0131 olabilir.<\/p>\n\n\n\n<p>Tarama b\u00fct\u00e7esi, bir arama motorunun herhangi bir web sitesinde tarayabilece\u011fi ve taramak istedi\u011fi maksimum sayfa say\u0131s\u0131d\u0131r. Google, tarama h\u0131z\u0131 s\u0131n\u0131r\u0131n\u0131 ve tarama talebini tartarak tarama b\u00fct\u00e7esini belirler. B\u00fct\u00e7ede baz\u0131 k\u00fc\u00e7\u00fck dalgalanmalar olabilir, ancak genel olarak nispeten tutarl\u0131 kal\u0131r. Bu kavram, k\u00fc\u00e7\u00fck web siteleriyle (birka\u00e7 y\u00fcz sayfa) alakal\u0131 de\u011fildir, \u00e7\u00fcnk\u00fc sitenin tamam\u0131 i\u00e7in tam bir tarama nispeten h\u0131zl\u0131 bir \u015fekilde ger\u00e7ekle\u015fir. Bununla birlikte, on binlerce veya daha fazla sayfadan olu\u015fan b\u00fcy\u00fck e-ticaret siteleri i\u00e7in, sitede hi\u00e7bir taray\u0131c\u0131 tuza\u011f\u0131 veya sonsuz tarama alan\u0131 bulunmad\u0131\u011f\u0131n\u0131n do\u011frulanmas\u0131 \u00f6nemlidir.<\/p>\n\n\n\n<p><strong>Bir e-ticaret sitesini tarama b\u00fct\u00e7esi i\u00e7in optimize etmek<\/strong>, Google&#8217;nin web taray\u0131c\u0131s\u0131n\u0131n belirli bir sayfay\u0131 ne s\u0131kl\u0131kta ziyaret etti\u011fini etkileyebilir ve bu nedenle, yeni veya g\u00fcncellenmi\u015f i\u00e7eri\u011fin Google arama sonu\u00e7lar\u0131nda daha h\u0131zl\u0131 g\u00f6r\u00fcnece\u011fi anlam\u0131na gelebilir.<\/p>\n\n\n\n<p>Siteniz&nbsp; bir e-ticaret sitesi ise ve otomatik olarak olu\u015fturulmu\u015f \u00e7ok say\u0131da URL&#8217;ye sahip binlerce sayfan\u0131z varsa, Google bu sayfalar\u0131n t\u00fcm\u00fcn\u00fc taramayabilir ve \u00e7ok say\u0131da potansiyel trafik ve g\u00f6r\u00fcn\u00fcrl\u00fckten mahrum kal\u0131rs\u0131n\u0131z.<\/p>\n\n\n\n<p>Google,<strong> &#8220;\u00e7ok say\u0131da d\u00fc\u015f\u00fck katma de\u011ferli URL&#8217;ye sahip olman\u0131n bir sitenin taranmas\u0131n\u0131 ve dizine eklenmesini olumsuz etkileyebilece\u011fini&#8221; <\/strong>belirtmi\u015ftir. Buras\u0131, bir robots.txt dosyas\u0131na sahip olman\u0131n, sitenizin tarama b\u00fct\u00e7esini etkileyen fakt\u00f6rlere yard\u0131mc\u0131 olabilece\u011fi yerdir.<\/p>\n\n\n\n<p>Bir arama motoru botu bir web sitesini tararken, sitenin hangi b\u00f6l\u00fcmlerinin dizine eklenmesi gerekti\u011fini belirlemek i\u00e7in robots.txt dosyas\u0131n\u0131 kullan\u0131r.<\/p>\n\n\n\n<p>Robots.txt dosyan\u0131z\u0131 botlar i\u00e7in bir k\u0131lavuz veya talimat k\u0131lavuzu gibi d\u00fc\u015f\u00fcn\u00fcn. Uymalar\u0131 gereken kurallar\u0131 olan bir rehberdir. Bu kurallar, taray\u0131c\u0131lara neleri g\u00f6r\u00fcnt\u00fcleyebileceklerini (site haritan\u0131zdaki sayfalar gibi) ve sitenizin hangi b\u00f6l\u00fcmlerinin k\u0131s\u0131tland\u0131\u011f\u0131n\u0131 s\u00f6yleyecektir.<\/p>\n\n\n\n<p>Dosyay\u0131, arama motorlar\u0131n\u0131n sitenizde olabildi\u011fince verimli bir \u015fekilde ve yaln\u0131zca \u00f6nemli sayfalar\u0131 taray\u0131p (giri\u015f, kay\u0131t veya te\u015fekk\u00fcr sayfalar\u0131 gibi sayfalar) zaman kaybetmeden sitenizde zaman ge\u00e7irmelerini sa\u011flayarak, sitelerinizin tarama b\u00fct\u00e7esini y\u00f6netmenize yard\u0131mc\u0131 olmas\u0131 i\u00e7in kullanabilirsiniz.&nbsp;<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Filtre Sayfalar\u0131n\u0131n Taranmas\u0131n\u0131 Engellemek<\/h2>\n\n\n\n<p>E-ticaret sitelerinin kategori sayfalar\u0131 olduk\u00e7a fazla filtre i\u00e7erebilir ve parametre i\u00e7eren filtre sayfalar\u0131 genellikle arama motorlar\u0131na indeksletilmez. Bu sayfalar\u0131 tarama i\u015fleminden hari\u00e7 tutmak faydal\u0131 olur.&nbsp;<\/p>\n\n\n\n<p>E-ticaret sitenizde bulunan fiyat filtresini ele alal\u0131m. <strong>siteadi.com\/?fiyat=150-250 <\/strong>gibi bir filtre URL\u2019niz varsa, onlarca filtre varyasyonu olu\u015fabilir ve sitenize gelen botlar\u0131n tarama limitini doldurabilir. Bunu engellemek i\u00e7in a\u015fa\u011f\u0131daki komutu yazabilirsiniz:<\/p>\n\n\n\n<p>User-agent: *<br>Disallow: \/*fiyat<\/p>\n\n\n\n<p>Anlam\u0131: T\u00fcm arama motoru botlar\u0131 sitemi ziyaret edebilir ama parametreli sayfalarda fiyat kelimesini i\u00e7eren sayfalar\u0131 ziyaret edemez.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Al\u0131\u015fveri\u015f Sayfalar\u0131n\u0131n Taranmas\u0131n\u0131 Engellemek<\/h2>\n\n\n\n<p>Sitenizde bulunan<strong> sepet, bo\u015f sepet, \u00f6deme, kargo se\u00e7imi <\/strong>ad\u0131mlar\u0131n\u0131n oldu\u011fu URL\u2019ler hem dinamik de\u011fi\u015fti\u011fi hem de dizine eklenmesinin bir faydas\u0131 olmad\u0131\u011f\u0131 i\u00e7in bu sayfalar\u0131n taranmas\u0131n\u0131 engelleyebilirsiniz.<\/p>\n\n\n\n<p>Sepet sayfam\u0131z\u0131n URL\u2019si: siteadi.com\/cart\/ilk-adim<\/p>\n\n\n\n<p>User-agent: *<br>Disallow: \/cart\/<\/p>\n\n\n\n<p>Anlam\u0131: Sitede bulunan \/cart\/ dizinini tarama.<\/p>\n\n\n\n<p>\u00d6deme sayfam\u0131z\u0131n URL\u2019si: siteadi.com\/payment\/<\/p>\n\n\n\n<p>User-agent: *<br>Disallow: \/payment\/<\/p>\n\n\n\n<p>Anlam\u0131: Sitede bulunan \/cart\/ dizinini tarama.<\/p>\n\n\n\n<p>Robots.txt dosyan\u0131z do\u011fru \u015fekilde optimize edilmemi\u015fse, web siteniz i\u00e7in b\u00fcy\u00fck SEO sorunlar\u0131na neden olabilir.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Robots.txt dosyas\u0131, bir sitenin k\u00f6k dizininde bulunan ve arama motoru taray\u0131c\u0131lar\u0131na y\u00f6nelik olarak hangi URL&#8217;lerin taranmas\u0131 ve hangilerinin taranmamas\u0131 gerekti\u011fi hakk\u0131nda bilgiler i\u00e7eren bir metin&#8230;<\/p>\n","protected":false},"author":11,"featured_media":5229,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[45],"tags":[],"class_list":["post-5227","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-internet"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v28.0 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Robot.txt Nedir? Robots TXT Dosyas\u0131 Nas\u0131l Olu\u015fturulur? - Niobe Hosting<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.niobehosting.com\/blog\/robot-txt\/\" \/>\n<meta property=\"og:locale\" content=\"tr_TR\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Robot.txt Nedir? Robots TXT Dosyas\u0131 Nas\u0131l Olu\u015fturulur? - Niobe Hosting\" \/>\n<meta property=\"og:description\" content=\"Robots.txt dosyas\u0131, bir sitenin k\u00f6k dizininde bulunan ve arama motoru taray\u0131c\u0131lar\u0131na y\u00f6nelik olarak hangi URL&#8217;lerin taranmas\u0131 ve hangilerinin taranmamas\u0131 gerekti\u011fi hakk\u0131nda bilgiler i\u00e7eren bir metin...\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.niobehosting.com\/blog\/robot-txt\/\" \/>\n<meta property=\"og:site_name\" content=\"Niobe Hosting\" \/>\n<meta property=\"article:published_time\" content=\"2021-03-16T18:24:45+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2021-05-03T08:47:11+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.niobehosting.com\/blog\/wp-content\/uploads\/robot-txt-nedir-robots-txt-dosyasi-nasil-olusturulur.png\" \/>\n\t<meta property=\"og:image:width\" content=\"720\" \/>\n\t<meta property=\"og:image:height\" content=\"400\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Sibel Ho\u015f\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Yazan:\" \/>\n\t<meta name=\"twitter:data1\" content=\"Sibel Ho\u015f\" \/>\n\t<meta name=\"twitter:label2\" content=\"Tahmini okuma s\u00fcresi\" \/>\n\t<meta name=\"twitter:data2\" content=\"11 dakika\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/www.niobehosting.com\\\/blog\\\/robot-txt\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.niobehosting.com\\\/blog\\\/robot-txt\\\/\"},\"author\":{\"name\":\"Sibel Ho\u015f\",\"@id\":\"https:\\\/\\\/www.niobehosting.com\\\/blog\\\/#\\\/schema\\\/person\\\/9edcd33ccd851c17242fa59c562c1f26\"},\"headline\":\"Robot.txt Nedir? Robots TXT Dosyas\u0131 Nas\u0131l Olu\u015fturulur?\",\"datePublished\":\"2021-03-16T18:24:45+00:00\",\"dateModified\":\"2021-05-03T08:47:11+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/www.niobehosting.com\\\/blog\\\/robot-txt\\\/\"},\"wordCount\":3482,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\\\/\\\/www.niobehosting.com\\\/blog\\\/#organization\"},\"image\":{\"@id\":\"https:\\\/\\\/www.niobehosting.com\\\/blog\\\/robot-txt\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/www.niobehosting.com\\\/blog\\\/wp-content\\\/uploads\\\/robot-txt-nedir-robots-txt-dosyasi-nasil-olusturulur.png\",\"articleSection\":[\"\u0130nternet\"],\"inLanguage\":\"tr\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/www.niobehosting.com\\\/blog\\\/robot-txt\\\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/www.niobehosting.com\\\/blog\\\/robot-txt\\\/\",\"url\":\"https:\\\/\\\/www.niobehosting.com\\\/blog\\\/robot-txt\\\/\",\"name\":\"Robot.txt Nedir? Robots TXT Dosyas\u0131 Nas\u0131l Olu\u015fturulur? - Niobe Hosting\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.niobehosting.com\\\/blog\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/www.niobehosting.com\\\/blog\\\/robot-txt\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/www.niobehosting.com\\\/blog\\\/robot-txt\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/www.niobehosting.com\\\/blog\\\/wp-content\\\/uploads\\\/robot-txt-nedir-robots-txt-dosyasi-nasil-olusturulur.png\",\"datePublished\":\"2021-03-16T18:24:45+00:00\",\"dateModified\":\"2021-05-03T08:47:11+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/www.niobehosting.com\\\/blog\\\/robot-txt\\\/#breadcrumb\"},\"inLanguage\":\"tr\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/www.niobehosting.com\\\/blog\\\/robot-txt\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"tr\",\"@id\":\"https:\\\/\\\/www.niobehosting.com\\\/blog\\\/robot-txt\\\/#primaryimage\",\"url\":\"https:\\\/\\\/www.niobehosting.com\\\/blog\\\/wp-content\\\/uploads\\\/robot-txt-nedir-robots-txt-dosyasi-nasil-olusturulur.png\",\"contentUrl\":\"https:\\\/\\\/www.niobehosting.com\\\/blog\\\/wp-content\\\/uploads\\\/robot-txt-nedir-robots-txt-dosyasi-nasil-olusturulur.png\",\"width\":720,\"height\":400},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/www.niobehosting.com\\\/blog\\\/robot-txt\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Anasayfa\",\"item\":\"https:\\\/\\\/www.niobehosting.com\\\/blog\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Robot.txt Nedir? Robots TXT Dosyas\u0131 Nas\u0131l Olu\u015fturulur?\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/www.niobehosting.com\\\/blog\\\/#website\",\"url\":\"https:\\\/\\\/www.niobehosting.com\\\/blog\\\/\",\"name\":\"Niobe Hosting\",\"description\":\"Niobe Hosting\",\"publisher\":{\"@id\":\"https:\\\/\\\/www.niobehosting.com\\\/blog\\\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/www.niobehosting.com\\\/blog\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"tr\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/www.niobehosting.com\\\/blog\\\/#organization\",\"name\":\"Niobe Hosting\",\"url\":\"https:\\\/\\\/www.niobehosting.com\\\/blog\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"tr\",\"@id\":\"https:\\\/\\\/www.niobehosting.com\\\/blog\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/www.niobehosting.com\\\/blog\\\/wp-content\\\/uploads\\\/blog-logo.png\",\"contentUrl\":\"https:\\\/\\\/www.niobehosting.com\\\/blog\\\/wp-content\\\/uploads\\\/blog-logo.png\",\"width\":332,\"height\":49,\"caption\":\"Niobe Hosting\"},\"image\":{\"@id\":\"https:\\\/\\\/www.niobehosting.com\\\/blog\\\/#\\\/schema\\\/logo\\\/image\\\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/www.niobehosting.com\\\/blog\\\/#\\\/schema\\\/person\\\/9edcd33ccd851c17242fa59c562c1f26\",\"name\":\"Sibel Ho\u015f\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"tr\",\"@id\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/f925f03d20ff0803ddb37c96d39d3793c085014ce56421050f24823c23db1c2a?s=96&d=mm&r=g\",\"url\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/f925f03d20ff0803ddb37c96d39d3793c085014ce56421050f24823c23db1c2a?s=96&d=mm&r=g\",\"contentUrl\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/f925f03d20ff0803ddb37c96d39d3793c085014ce56421050f24823c23db1c2a?s=96&d=mm&r=g\",\"caption\":\"Sibel Ho\u015f\"},\"sameAs\":[\"https:\\\/\\\/www.instagram.com\\\/bismoothie\\\/\",\"https:\\\/\\\/www.linkedin.com\\\/in\\\/sibelhos\\\/\"],\"url\":\"https:\\\/\\\/www.niobehosting.com\\\/blog\\\/author\\\/sibel\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Robot.txt Nedir? Robots TXT Dosyas\u0131 Nas\u0131l Olu\u015fturulur? - Niobe Hosting","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.niobehosting.com\/blog\/robot-txt\/","og_locale":"tr_TR","og_type":"article","og_title":"Robot.txt Nedir? Robots TXT Dosyas\u0131 Nas\u0131l Olu\u015fturulur? - Niobe Hosting","og_description":"Robots.txt dosyas\u0131, bir sitenin k\u00f6k dizininde bulunan ve arama motoru taray\u0131c\u0131lar\u0131na y\u00f6nelik olarak hangi URL&#8217;lerin taranmas\u0131 ve hangilerinin taranmamas\u0131 gerekti\u011fi hakk\u0131nda bilgiler i\u00e7eren bir metin...","og_url":"https:\/\/www.niobehosting.com\/blog\/robot-txt\/","og_site_name":"Niobe Hosting","article_published_time":"2021-03-16T18:24:45+00:00","article_modified_time":"2021-05-03T08:47:11+00:00","og_image":[{"width":720,"height":400,"url":"https:\/\/www.niobehosting.com\/blog\/wp-content\/uploads\/robot-txt-nedir-robots-txt-dosyasi-nasil-olusturulur.png","type":"image\/png"}],"author":"Sibel Ho\u015f","twitter_card":"summary_large_image","twitter_misc":{"Yazan:":"Sibel Ho\u015f","Tahmini okuma s\u00fcresi":"11 dakika"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.niobehosting.com\/blog\/robot-txt\/#article","isPartOf":{"@id":"https:\/\/www.niobehosting.com\/blog\/robot-txt\/"},"author":{"name":"Sibel Ho\u015f","@id":"https:\/\/www.niobehosting.com\/blog\/#\/schema\/person\/9edcd33ccd851c17242fa59c562c1f26"},"headline":"Robot.txt Nedir? Robots TXT Dosyas\u0131 Nas\u0131l Olu\u015fturulur?","datePublished":"2021-03-16T18:24:45+00:00","dateModified":"2021-05-03T08:47:11+00:00","mainEntityOfPage":{"@id":"https:\/\/www.niobehosting.com\/blog\/robot-txt\/"},"wordCount":3482,"commentCount":0,"publisher":{"@id":"https:\/\/www.niobehosting.com\/blog\/#organization"},"image":{"@id":"https:\/\/www.niobehosting.com\/blog\/robot-txt\/#primaryimage"},"thumbnailUrl":"https:\/\/www.niobehosting.com\/blog\/wp-content\/uploads\/robot-txt-nedir-robots-txt-dosyasi-nasil-olusturulur.png","articleSection":["\u0130nternet"],"inLanguage":"tr","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.niobehosting.com\/blog\/robot-txt\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.niobehosting.com\/blog\/robot-txt\/","url":"https:\/\/www.niobehosting.com\/blog\/robot-txt\/","name":"Robot.txt Nedir? Robots TXT Dosyas\u0131 Nas\u0131l Olu\u015fturulur? - Niobe Hosting","isPartOf":{"@id":"https:\/\/www.niobehosting.com\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.niobehosting.com\/blog\/robot-txt\/#primaryimage"},"image":{"@id":"https:\/\/www.niobehosting.com\/blog\/robot-txt\/#primaryimage"},"thumbnailUrl":"https:\/\/www.niobehosting.com\/blog\/wp-content\/uploads\/robot-txt-nedir-robots-txt-dosyasi-nasil-olusturulur.png","datePublished":"2021-03-16T18:24:45+00:00","dateModified":"2021-05-03T08:47:11+00:00","breadcrumb":{"@id":"https:\/\/www.niobehosting.com\/blog\/robot-txt\/#breadcrumb"},"inLanguage":"tr","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.niobehosting.com\/blog\/robot-txt\/"]}]},{"@type":"ImageObject","inLanguage":"tr","@id":"https:\/\/www.niobehosting.com\/blog\/robot-txt\/#primaryimage","url":"https:\/\/www.niobehosting.com\/blog\/wp-content\/uploads\/robot-txt-nedir-robots-txt-dosyasi-nasil-olusturulur.png","contentUrl":"https:\/\/www.niobehosting.com\/blog\/wp-content\/uploads\/robot-txt-nedir-robots-txt-dosyasi-nasil-olusturulur.png","width":720,"height":400},{"@type":"BreadcrumbList","@id":"https:\/\/www.niobehosting.com\/blog\/robot-txt\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Anasayfa","item":"https:\/\/www.niobehosting.com\/blog\/"},{"@type":"ListItem","position":2,"name":"Robot.txt Nedir? Robots TXT Dosyas\u0131 Nas\u0131l Olu\u015fturulur?"}]},{"@type":"WebSite","@id":"https:\/\/www.niobehosting.com\/blog\/#website","url":"https:\/\/www.niobehosting.com\/blog\/","name":"Niobe Hosting","description":"Niobe Hosting","publisher":{"@id":"https:\/\/www.niobehosting.com\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.niobehosting.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"tr"},{"@type":"Organization","@id":"https:\/\/www.niobehosting.com\/blog\/#organization","name":"Niobe Hosting","url":"https:\/\/www.niobehosting.com\/blog\/","logo":{"@type":"ImageObject","inLanguage":"tr","@id":"https:\/\/www.niobehosting.com\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/www.niobehosting.com\/blog\/wp-content\/uploads\/blog-logo.png","contentUrl":"https:\/\/www.niobehosting.com\/blog\/wp-content\/uploads\/blog-logo.png","width":332,"height":49,"caption":"Niobe Hosting"},"image":{"@id":"https:\/\/www.niobehosting.com\/blog\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/www.niobehosting.com\/blog\/#\/schema\/person\/9edcd33ccd851c17242fa59c562c1f26","name":"Sibel Ho\u015f","image":{"@type":"ImageObject","inLanguage":"tr","@id":"https:\/\/secure.gravatar.com\/avatar\/f925f03d20ff0803ddb37c96d39d3793c085014ce56421050f24823c23db1c2a?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/f925f03d20ff0803ddb37c96d39d3793c085014ce56421050f24823c23db1c2a?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/f925f03d20ff0803ddb37c96d39d3793c085014ce56421050f24823c23db1c2a?s=96&d=mm&r=g","caption":"Sibel Ho\u015f"},"sameAs":["https:\/\/www.instagram.com\/bismoothie\/","https:\/\/www.linkedin.com\/in\/sibelhos\/"],"url":"https:\/\/www.niobehosting.com\/blog\/author\/sibel\/"}]}},"_links":{"self":[{"href":"https:\/\/www.niobehosting.com\/blog\/wp-json\/wp\/v2\/posts\/5227","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.niobehosting.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.niobehosting.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.niobehosting.com\/blog\/wp-json\/wp\/v2\/users\/11"}],"replies":[{"embeddable":true,"href":"https:\/\/www.niobehosting.com\/blog\/wp-json\/wp\/v2\/comments?post=5227"}],"version-history":[{"count":0,"href":"https:\/\/www.niobehosting.com\/blog\/wp-json\/wp\/v2\/posts\/5227\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.niobehosting.com\/blog\/wp-json\/wp\/v2\/media\/5229"}],"wp:attachment":[{"href":"https:\/\/www.niobehosting.com\/blog\/wp-json\/wp\/v2\/media?parent=5227"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.niobehosting.com\/blog\/wp-json\/wp\/v2\/categories?post=5227"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.niobehosting.com\/blog\/wp-json\/wp\/v2\/tags?post=5227"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}