{"id":3502,"date":"2020-01-07T05:07:46","date_gmt":"2020-01-07T02:07:46","guid":{"rendered":"https:\/\/www.lf.com.tr\/?p=3502"},"modified":"2025-11-18T09:23:16","modified_gmt":"2025-11-18T06:23:16","slug":"robots-txt-nedir-ve-ne-ise-yarar","status":"publish","type":"post","link":"https:\/\/www.lf.com.tr\/tr\/robots-txt-nedir-ve-ne-ise-yarar\/","title":{"rendered":"Robots.txt Nedir ve Ne \u0130\u015fe Yarar?"},"content":{"rendered":"

Arama motorlar\u0131 botlar\u0131, web sitelerinin sayfalar\u0131n\u0131 tararlar ve ar\u015fivlerler. Ancak zaman zaman web sitemize ait olan her sayfan\u0131n arama motoru botlar\u0131 taraf\u0131ndan bulunmas\u0131n\u0131 istemeyiz. \u0130\u015fte tam da burada Robots.txt metin dosyalar\u0131 devreye girer. Arama motorlar\u0131na kapatmak istedi\u011finiz her sayfa ya da uzant\u0131y\u0131 Robots.txt dosyalar\u0131 ile gizleyebiliriz. Ancak bu dosya olu\u015fturulurken dikkat edilmesi de gerekir. \u00c7\u00fcnk\u00fc yanl\u0131\u015f olu\u015fturulan bir Robots.txt dosyas\u0131 sitenizin \u00f6nemli sayfalar\u0131n\u0131, arama motorlar\u0131 taraf\u0131ndan bulunmas\u0131n\u0131 istedi\u011finiz sayfalar\u0131 da eri\u015fime kapatabilir. SEO a\u00e7\u0131s\u0131ndan \u00f6nemli olan bu duruma nas\u0131l dikkat edilir, bu dosya nas\u0131l olu\u015fturulur gelin hep birlikte inceleyelim;<\/p>\n

Robots.txt Nedir?<\/strong><\/span><\/h2>\n

Robots.txt metin dosyas\u0131, herhangi bir arama motoru i\u00e7in sitenizin hangi sayfa ya da b\u00f6l\u00fcmlerinin dizine eklenece\u011fini ya da eklenemeyece\u011fini s\u00f6yleyen basit bir komut dosyas\u0131d\u0131r. Arama motorlar\u0131 taraf\u0131ndan g\u00f6revlendirilen, \u00f6r\u00fcmcek ad\u0131 da verilen botlar sitenize geldi\u011finde ilk \u00f6nce bu dosyay\u0131 tarar ve bu dosyadaki komutlara g\u00f6re sitenin izin verilen b\u00f6l\u00fcmlerini dizine ekler ya da eklemez. E\u011fer Robots.txt dosyan\u0131z do\u011fru bir \u015fekilde haz\u0131rland\u0131ysa, arama motoru robotlar\u0131 yazd\u0131\u011f\u0131n\u0131z bu\u00a0 komutlara uygun bir \u015fekilde tarama i\u015flemlerini ger\u00e7ekle\u015ftirecektir.<\/p>\n

 <\/p>\n

G\u00f6rd\u00fc\u011f\u00fcn\u00fcz \u00fczere Robots.txt dosyas\u0131 her web sitesinde mutlaka olmas\u0131 gereken bir dosyad\u0131r, tabii do\u011fru haz\u0131rlanm\u0131\u015f bir \u015fekilde.<\/p>\n

 <\/p>\n

\"robots-txt-ornek\"<\/p>\n

Robots.txt Dosyas\u0131 Nas\u0131l Olu\u015fturulur?<\/strong><\/span><\/h2>\n

Robots.txt komut dosyas\u0131 belli standartlar \u00e7er\u00e7evesinde haz\u0131rlanmal\u0131 ve web sitenizin k\u00f6k dizinine bu \u015fekilde y\u00fcklenmelidir. \u00d6ncelikle web sitenizin k\u00f6k dizininde robots.txt dosyan\u0131z\u0131n bulunmas\u0131 bir gerekliliktir ve dosya olu\u015fturulurken bir alt klas\u00f6rde ya da farkl\u0131 sayfalarda yer almamas\u0131 gerekir.<\/p>\n

 <\/p>\n

\u00d6rnek bir\u00a0do\u011fru kullan\u0131m<\/u>\u00a0g\u00f6stermemiz gerekirse;\u00a0https:\/\/www.ornek.com\/robots.txt<\/em>\u00a0\u015feklinde olabilece\u011finiz s\u00f6yleyebiliriz. Ancak\u00a0https:\/\/www.ornek.com\/main\/robots.txt<\/em>\u00a0kullan\u0131m\u0131 robots.txt i\u00e7in\u00a0yanl\u0131\u015f<\/u>\u00a0bir kullan\u0131m olacakt\u0131r.<\/p>\n

 <\/p>\n

Robots.txt dosyas\u0131 d\u00fcz metinden olu\u015fmal\u0131d\u0131r ve UTF-8 karakter kodlamalar\u0131 kullan\u0131lmal\u0131d\u0131r. Dosya s\u00fcrekli g\u00fcncellenebilir oldu\u011fu i\u00e7in istedi\u011finiz zaman \u00fczerinde \u00e7al\u0131\u015fabilece\u011finiz bir \u015fekilde tutmal\u0131, gizlememeli ya da kald\u0131rmamal\u0131s\u0131n\u0131z.<\/p>\n

Robots.txt Komutlar\u0131<\/strong><\/span><\/h2>\n

Robots.txt i\u00e7erisinde kullan\u0131lan baz\u0131 standartla\u015fm\u0131\u015f komutlar bulunmaktad\u0131r. Bunlar temel olarak \u015fu \u015fekilde s\u0131ralanabilir;<\/p>\n

 <\/p>\n

Grup Komutlar\u0131;\u00a0<\/strong>Arama motorlar\u0131 engelleme ya da izin verme, dizin engelleme\/izin verme, sayfa engelleme\/izin verme ve tarama geciktirmesi<\/p>\n

 <\/p>\n

Grup D\u0131\u015f\u0131 Komutlar;\u00a0<\/strong>Site haritas\u0131 belirtme<\/p>\n

 <\/p>\n

Robots.txt Dosyas\u0131nda Kullan\u0131labilen \u00d6zel Komutlar<\/strong><\/p>\n

 <\/p>\n

 <\/p>\n

Grup Komutlar\u0131<\/strong><\/span><\/h2>\n

Dosya i\u00e7erisinde net ve kesin ifadelerden olu\u015fmas\u0131 gereken grup komutlar\u0131nda e\u011fer bir yaz\u0131m yanl\u0131\u015f\u0131 yap\u0131l\u0131rsa bu komut dikkate al\u0131nmaz. Yine komutlar i\u00e7erisinde b\u00fcy\u00fck-k\u00fc\u00e7\u00fck harf kullan\u0131m\u0131na, \u00f6zel karakter ayr\u0131mlar\u0131na dikkat edilmesi gerekir. En \u00e7ok kullan\u0131lan grup komutlar\u0131 \u015funlard\u0131r;<\/p>\n

 <\/p>\n

User-agent<\/strong><\/span><\/h3>\n

 <\/p>\n

Bu komut hangi arama motorunun web sitenizi taramas\u0131na izin verip vermeyece\u011fini belirtmesine yard\u0131mc\u0131 olur. T\u00fcm taray\u0131c\u0131lara tarama izni vermek isterseniz e\u011fer * i\u015faretini kullanabilirsiniz. Ya da sadece bir taray\u0131c\u0131 i\u00e7in izin verecekseniz taray\u0131c\u0131 ad\u0131n\u0131 belirtebilirsiniz.<\/p>\n

 <\/p>\n

\"robots-txt-disallow\"<\/p>\n

 <\/p>\n

Dizin Engelleme \/ \u0130zin verme<\/strong><\/span><\/h3>\n

 <\/p>\n

Disallow ya da Allow komutlar\u0131 kullan\u0131larak sitedeki belli dizinlere eri\u015fim kald\u0131r\u0131labilir ya da izin verilebilir. Bu eri\u015fim izinleri sadece arama motorlar\u0131na y\u00f6nelik olup kullan\u0131c\u0131lar komut verilen dizine ula\u015fmaya devam edebileceklerdir.<\/p>\n

 <\/p>\n

\"robots-txt-disallow\"<\/p>\n

 <\/p>\n

Yukar\u0131daki komut kullan\u0131ld\u0131\u011f\u0131nda arama motorunuz\u00a0https:\/\/www.ornek.com\/tarama\/<\/em>\u00a0URL\u2019sine sahip olan \u201ctarama\u201d dizinini taramayacak ve yine bu dizin i\u00e7erisinde bulunan sayfalar\u0131 dizine eklemeyecektir.<\/p>\n

 <\/p>\n

Sayfa Engelleme \/ \u0130zin Verme<\/strong><\/span><\/h3>\n

 <\/p>\n

Ayn\u0131 \u015fekilde disallow ya da allow komutlar\u0131 kullan\u0131larak belirli sayfalara eri\u015fim izninin verilmesi ya da verilmemesidir.<\/p>\n

 <\/p>\n

 <\/p>\n

 <\/p>\n

\"robots-txt\"<\/p>\n

 <\/p>\n

Yukar\u0131daki komut kullan\u0131ld\u0131\u011f\u0131nda,\u00a0https:\/\/www.ornek.com\/ozel-tarama.html<\/em>\u00a0URL\u2019sine sahip olan sayfadaki i\u00e7erik taranmayacak ve dizine eklenmeyecektir.<\/p>\n

 <\/p>\n

Allow komutu da disallow komutunun tam tersi olarak i\u015flem yapar ve Disallow komutu kullan\u0131m\u0131n\u0131 ge\u00e7ersiz k\u0131lar.<\/p>\n

 <\/p>\n

\"robots-txt-allow\"<\/p>\n

 <\/p>\n

Yukar\u0131daki gibi Allow ve Disallow komutlar\u0131 birlikte kullan\u0131l\u0131rsa tarama dosyas\u0131 i\u00e7erisindeki demo.html sayfas\u0131 hari\u00e7 di\u011fer dosyalar\u0131n taramas\u0131n\u0131 kapatmak anlam\u0131na gelecektir. Bu \u00f6rnekten de anla\u015f\u0131laca\u011f\u0131 \u00fczere Google botlar\u0131 \u00f6nce Allow sonras\u0131nda Disallow komutunu i\u015fleme alacakt\u0131r.<\/p>\n

Tarama Geciktirme (Crawl-delay)\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/strong><\/span><\/h2>\n

Bu komut sayesinde arama motoru botlar\u0131n\u0131n tarama i\u00e7in harcayaca\u011f\u0131 zaman\u0131 s\u0131n\u0131rland\u0131rman\u0131z m\u00fcmk\u00fcn. \u00d6rne\u011fin ge\u00e7 a\u00e7\u0131lan bir web siteniz varsa ve arama motoru botlar\u0131n\u0131n sayfan\u0131za k\u00f6t\u00fc puan vermesini istemiyorsan\u0131z a\u00e7\u0131l\u0131\u015f s\u00fcresini buna g\u00f6re ayarlayabilirsiniz.<\/p>\n

 <\/p>\n

\"robots-txt-tarama-geciktirme\"<\/p>\n

Grup D\u0131\u015f\u0131 Komutlar<\/strong><\/span><\/h2>\n

Grup d\u0131\u015f\u0131 olarak ge\u00e7en en \u00f6nemli komut sitemap.xml komutudur. Bu komut, arama motoru botlar\u0131na \u00f6nemli bilgiler verdi\u011fi i\u00e7in, sitenize gelen botlar\u0131n tarad\u0131\u011f\u0131 ilk yer olarak s\u00f6ylenebilir. Bu nedenle sitenizde sitemap.xml dosyas\u0131 bulunmazsa bu bir dezavantaj yaratabilir. Kullan\u0131m\u0131 da olduk\u00e7a basit olan bu komut i\u00e7in sitemap.xml dosyas\u0131n\u0131n URL\u2019sini do\u011fru yazman\u0131z yeterlidir.<\/p>\n

 <\/p>\n

\"sitemap-xml\"<\/p>\n

Robots.txt Dosyas\u0131nda Kullan\u0131labilen \u00d6zel Komutlar<\/strong><\/span><\/h2>\n

Robots.txt komut dosyas\u0131na ekleyece\u011finiz \u00f6zel ifadeler ile belli bir karakteri ya da URL uzant\u0131s\u0131n\u0131 i\u00e7eren sayfalar\u0131 da istedi\u011finiz arama motoru robotlar\u0131na kapatabilirsiniz.<\/p>\n

 <\/p>\n

\u00d6rne\u011fin;<\/p>\n

 <\/p>\n

\"robots-txt\"<\/p>\n

 <\/p>\n

\"robots-txt\"<\/p>\n

 <\/p>\n

Son olarak s\u00f6ylemeliyiz ki; e\u011fer web sitenizi yeni kuruyor ya da tasar\u0131m\u0131n\u0131 de\u011fi\u015ftiriyorsan\u0131z mutlaka ilk kontrol etmeniz gereken \u015fey robots.txt dosyas\u0131d\u0131r. Hali haz\u0131rda bir dosyan\u0131z yoksa da olu\u015fturmal\u0131 ve web sitenize eklemelisiniz. \u00d6zellikle SEO a\u00e7\u0131s\u0131ndan kullan\u0131lmamas\u0131 b\u00fcy\u00fck dezavantaj olan Robots.txt dosyas\u0131 yoksa web sitenizde, arama motoru botlar\u0131 noindex etiketi olmayan sayfalar\u0131 tarayacaklard\u0131r. Taranmas\u0131n\u0131 istemedi\u011finiz her sayfa ve klas\u00f6re noindex etiketini tek tek yerle\u015ftirmek yerine Robots.txt dosyas\u0131 olu\u015fturarak zaman kazanabilir ve sitenizin istedi\u011finiz arama motoru dizinine istedi\u011finiz \u015fekilde eklenmesini sa\u011flayabilirsiniz.<\/p>\n

 <\/p>\n

Siteniz i\u00e7in SEO Analizi yapma konusunda fikir sahibi olmak i\u00e7in\u00a0buraya<\/a>\u00a0t\u0131klayarak yaz\u0131m\u0131z\u0131 inceleyebilirsiniz.<\/p>","protected":false},"excerpt":{"rendered":"

Arama motorlar\u0131 botlar\u0131, web sitelerinin sayfalar\u0131n\u0131 tararlar ve ar\u015fivlerler. Ancak zaman zaman web sitemize ait olan her sayfan\u0131n arama motoru botlar\u0131 taraf\u0131ndan bulunmas\u0131n\u0131 istemeyiz. \u0130\u015fte tam da burada Robots.txt metin dosyalar\u0131 devreye girer. Arama motorlar\u0131na kapatmak istedi\u011finiz her sayfa ya da uzant\u0131y\u0131 Robots.txt dosyalar\u0131 ile gizleyebiliriz. Ancak bu dosya olu\u015fturulurken dikkat edilmesi de gerekir. \u00c7\u00fcnk\u00fc…<\/p>","protected":false},"author":1,"featured_media":4124,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-3502","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-blog"],"yoast_head":"\nRobots.txt Nedir ve Ne \u0130\u015fe Yarar? - LF Dijital - Google Kocaeli - Web Tasar\u0131m - Yaz\u0131l\u0131m - Dijital Pazarlama - SEO<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.lf.com.tr\/tr\/robots-txt-nedir-ve-ne-ise-yarar\/\" \/>\n<meta property=\"og:locale\" content=\"tr_TR\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Robots.txt Nedir ve Ne \u0130\u015fe Yarar? - LF Dijital - Google Kocaeli - Web Tasar\u0131m - Yaz\u0131l\u0131m - Dijital Pazarlama - SEO\" \/>\n<meta property=\"og:description\" content=\"Arama motorlar\u0131 botlar\u0131, web sitelerinin sayfalar\u0131n\u0131 tararlar ve ar\u015fivlerler. Ancak zaman zaman web sitemize ait olan her sayfan\u0131n arama motoru botlar\u0131 taraf\u0131ndan bulunmas\u0131n\u0131 istemeyiz. \u0130\u015fte tam da burada Robots.txt metin dosyalar\u0131 devreye girer. Arama motorlar\u0131na kapatmak istedi\u011finiz her sayfa ya da uzant\u0131y\u0131 Robots.txt dosyalar\u0131 ile gizleyebiliriz. Ancak bu dosya olu\u015fturulurken dikkat edilmesi de gerekir. \u00c7\u00fcnk\u00fc...\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.lf.com.tr\/tr\/robots-txt-nedir-ve-ne-ise-yarar\/\" \/>\n<meta property=\"og:site_name\" content=\"LF Dijital - Google Kocaeli - Web Tasar\u0131m - Yaz\u0131l\u0131m - Dijital Pazarlama - SEO\" \/>\n<meta property=\"article:published_time\" content=\"2020-01-07T02:07:46+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-11-18T06:23:16+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.lf.com.tr\/wp-content\/uploads\/2020\/01\/lf_site-1.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"400\" \/>\n\t<meta property=\"og:image:height\" content=\"300\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"lfdigital\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Yazan:\" \/>\n\t<meta name=\"twitter:data1\" content=\"lfdigital\" \/>\n\t<meta name=\"twitter:label2\" content=\"Tahmini okuma s\u00fcresi\" \/>\n\t<meta name=\"twitter:data2\" content=\"5 dakika\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/www.lf.com.tr\\\/robots-txt-nedir-ve-ne-ise-yarar\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.lf.com.tr\\\/robots-txt-nedir-ve-ne-ise-yarar\\\/\"},\"author\":{\"name\":\"lfdigital\",\"@id\":\"https:\\\/\\\/www.lf.com.tr\\\/#\\\/schema\\\/person\\\/f01d2cc7ecbefa065757f1c757ef2457\"},\"headline\":\"Robots.txt Nedir ve Ne \u0130\u015fe Yarar?\",\"datePublished\":\"2020-01-07T02:07:46+00:00\",\"dateModified\":\"2025-11-18T06:23:16+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/www.lf.com.tr\\\/robots-txt-nedir-ve-ne-ise-yarar\\\/\"},\"wordCount\":1058,\"image\":{\"@id\":\"https:\\\/\\\/www.lf.com.tr\\\/robots-txt-nedir-ve-ne-ise-yarar\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/www.lf.com.tr\\\/wp-content\\\/uploads\\\/2020\\\/01\\\/lf_site-1.jpg\",\"articleSection\":[\"Blog\"],\"inLanguage\":\"tr\"},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/www.lf.com.tr\\\/robots-txt-nedir-ve-ne-ise-yarar\\\/\",\"url\":\"https:\\\/\\\/www.lf.com.tr\\\/robots-txt-nedir-ve-ne-ise-yarar\\\/\",\"name\":\"Robots.txt Nedir ve Ne \u0130\u015fe Yarar? - LF Dijital - Google Kocaeli - Web Tasar\u0131m - Yaz\u0131l\u0131m - Dijital Pazarlama - SEO\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.lf.com.tr\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/www.lf.com.tr\\\/robots-txt-nedir-ve-ne-ise-yarar\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/www.lf.com.tr\\\/robots-txt-nedir-ve-ne-ise-yarar\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/www.lf.com.tr\\\/wp-content\\\/uploads\\\/2020\\\/01\\\/lf_site-1.jpg\",\"datePublished\":\"2020-01-07T02:07:46+00:00\",\"dateModified\":\"2025-11-18T06:23:16+00:00\",\"author\":{\"@id\":\"https:\\\/\\\/www.lf.com.tr\\\/#\\\/schema\\\/person\\\/f01d2cc7ecbefa065757f1c757ef2457\"},\"breadcrumb\":{\"@id\":\"https:\\\/\\\/www.lf.com.tr\\\/robots-txt-nedir-ve-ne-ise-yarar\\\/#breadcrumb\"},\"inLanguage\":\"tr\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/www.lf.com.tr\\\/robots-txt-nedir-ve-ne-ise-yarar\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"tr\",\"@id\":\"https:\\\/\\\/www.lf.com.tr\\\/robots-txt-nedir-ve-ne-ise-yarar\\\/#primaryimage\",\"url\":\"https:\\\/\\\/www.lf.com.tr\\\/wp-content\\\/uploads\\\/2020\\\/01\\\/lf_site-1.jpg\",\"contentUrl\":\"https:\\\/\\\/www.lf.com.tr\\\/wp-content\\\/uploads\\\/2020\\\/01\\\/lf_site-1.jpg\",\"width\":400,\"height\":300},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/www.lf.com.tr\\\/robots-txt-nedir-ve-ne-ise-yarar\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Ana sayfa\",\"item\":\"https:\\\/\\\/www.lf.com.tr\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Robots.txt Nedir ve Ne \u0130\u015fe Yarar?\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/www.lf.com.tr\\\/#website\",\"url\":\"https:\\\/\\\/www.lf.com.tr\\\/\",\"name\":\"LF Dijital - Google Kocaeli - Web Tasar\u0131m - Yaz\u0131l\u0131m - Dijital Pazarlama - SEO\",\"description\":\"Kocaeli web tasar\u0131m firmalar\u0131\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/www.lf.com.tr\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"tr\"},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/www.lf.com.tr\\\/#\\\/schema\\\/person\\\/f01d2cc7ecbefa065757f1c757ef2457\",\"name\":\"lfdigital\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"tr\",\"@id\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/d6d1d054ded44c6b71c5ad207ee6ab9769ab199182963c14fbd52da73676c654?s=96&d=mm&r=g\",\"url\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/d6d1d054ded44c6b71c5ad207ee6ab9769ab199182963c14fbd52da73676c654?s=96&d=mm&r=g\",\"contentUrl\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/d6d1d054ded44c6b71c5ad207ee6ab9769ab199182963c14fbd52da73676c654?s=96&d=mm&r=g\",\"caption\":\"lfdigital\"},\"url\":\"https:\\\/\\\/www.lf.com.tr\\\/tr\\\/author\\\/lfdigital\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Robots.txt Nedir ve Ne \u0130\u015fe Yarar? - LF Dijital - Google Kocaeli - Web Tasar\u0131m - Yaz\u0131l\u0131m - Dijital Pazarlama - SEO","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.lf.com.tr\/tr\/robots-txt-nedir-ve-ne-ise-yarar\/","og_locale":"tr_TR","og_type":"article","og_title":"Robots.txt Nedir ve Ne \u0130\u015fe Yarar? - LF Dijital - Google Kocaeli - Web Tasar\u0131m - Yaz\u0131l\u0131m - Dijital Pazarlama - SEO","og_description":"Arama motorlar\u0131 botlar\u0131, web sitelerinin sayfalar\u0131n\u0131 tararlar ve ar\u015fivlerler. Ancak zaman zaman web sitemize ait olan her sayfan\u0131n arama motoru botlar\u0131 taraf\u0131ndan bulunmas\u0131n\u0131 istemeyiz. \u0130\u015fte tam da burada Robots.txt metin dosyalar\u0131 devreye girer. Arama motorlar\u0131na kapatmak istedi\u011finiz her sayfa ya da uzant\u0131y\u0131 Robots.txt dosyalar\u0131 ile gizleyebiliriz. Ancak bu dosya olu\u015fturulurken dikkat edilmesi de gerekir. \u00c7\u00fcnk\u00fc...","og_url":"https:\/\/www.lf.com.tr\/tr\/robots-txt-nedir-ve-ne-ise-yarar\/","og_site_name":"LF Dijital - Google Kocaeli - Web Tasar\u0131m - Yaz\u0131l\u0131m - Dijital Pazarlama - SEO","article_published_time":"2020-01-07T02:07:46+00:00","article_modified_time":"2025-11-18T06:23:16+00:00","og_image":[{"width":400,"height":300,"url":"https:\/\/www.lf.com.tr\/wp-content\/uploads\/2020\/01\/lf_site-1.jpg","type":"image\/jpeg"}],"author":"lfdigital","twitter_card":"summary_large_image","twitter_misc":{"Yazan:":"lfdigital","Tahmini okuma s\u00fcresi":"5 dakika"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.lf.com.tr\/robots-txt-nedir-ve-ne-ise-yarar\/#article","isPartOf":{"@id":"https:\/\/www.lf.com.tr\/robots-txt-nedir-ve-ne-ise-yarar\/"},"author":{"name":"lfdigital","@id":"https:\/\/www.lf.com.tr\/#\/schema\/person\/f01d2cc7ecbefa065757f1c757ef2457"},"headline":"Robots.txt Nedir ve Ne \u0130\u015fe Yarar?","datePublished":"2020-01-07T02:07:46+00:00","dateModified":"2025-11-18T06:23:16+00:00","mainEntityOfPage":{"@id":"https:\/\/www.lf.com.tr\/robots-txt-nedir-ve-ne-ise-yarar\/"},"wordCount":1058,"image":{"@id":"https:\/\/www.lf.com.tr\/robots-txt-nedir-ve-ne-ise-yarar\/#primaryimage"},"thumbnailUrl":"https:\/\/www.lf.com.tr\/wp-content\/uploads\/2020\/01\/lf_site-1.jpg","articleSection":["Blog"],"inLanguage":"tr"},{"@type":"WebPage","@id":"https:\/\/www.lf.com.tr\/robots-txt-nedir-ve-ne-ise-yarar\/","url":"https:\/\/www.lf.com.tr\/robots-txt-nedir-ve-ne-ise-yarar\/","name":"Robots.txt Nedir ve Ne \u0130\u015fe Yarar? - LF Dijital - Google Kocaeli - Web Tasar\u0131m - Yaz\u0131l\u0131m - Dijital Pazarlama - SEO","isPartOf":{"@id":"https:\/\/www.lf.com.tr\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.lf.com.tr\/robots-txt-nedir-ve-ne-ise-yarar\/#primaryimage"},"image":{"@id":"https:\/\/www.lf.com.tr\/robots-txt-nedir-ve-ne-ise-yarar\/#primaryimage"},"thumbnailUrl":"https:\/\/www.lf.com.tr\/wp-content\/uploads\/2020\/01\/lf_site-1.jpg","datePublished":"2020-01-07T02:07:46+00:00","dateModified":"2025-11-18T06:23:16+00:00","author":{"@id":"https:\/\/www.lf.com.tr\/#\/schema\/person\/f01d2cc7ecbefa065757f1c757ef2457"},"breadcrumb":{"@id":"https:\/\/www.lf.com.tr\/robots-txt-nedir-ve-ne-ise-yarar\/#breadcrumb"},"inLanguage":"tr","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.lf.com.tr\/robots-txt-nedir-ve-ne-ise-yarar\/"]}]},{"@type":"ImageObject","inLanguage":"tr","@id":"https:\/\/www.lf.com.tr\/robots-txt-nedir-ve-ne-ise-yarar\/#primaryimage","url":"https:\/\/www.lf.com.tr\/wp-content\/uploads\/2020\/01\/lf_site-1.jpg","contentUrl":"https:\/\/www.lf.com.tr\/wp-content\/uploads\/2020\/01\/lf_site-1.jpg","width":400,"height":300},{"@type":"BreadcrumbList","@id":"https:\/\/www.lf.com.tr\/robots-txt-nedir-ve-ne-ise-yarar\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Ana sayfa","item":"https:\/\/www.lf.com.tr\/"},{"@type":"ListItem","position":2,"name":"Robots.txt Nedir ve Ne \u0130\u015fe Yarar?"}]},{"@type":"WebSite","@id":"https:\/\/www.lf.com.tr\/#website","url":"https:\/\/www.lf.com.tr\/","name":"LF Dijital - Google Kocaeli - Web Tasar\u0131m - Yaz\u0131l\u0131m - Dijital Pazarlama - SEO","description":"Kocaeli web tasar\u0131m firmalar\u0131","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.lf.com.tr\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"tr"},{"@type":"Person","@id":"https:\/\/www.lf.com.tr\/#\/schema\/person\/f01d2cc7ecbefa065757f1c757ef2457","name":"lfdigital","image":{"@type":"ImageObject","inLanguage":"tr","@id":"https:\/\/secure.gravatar.com\/avatar\/d6d1d054ded44c6b71c5ad207ee6ab9769ab199182963c14fbd52da73676c654?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/d6d1d054ded44c6b71c5ad207ee6ab9769ab199182963c14fbd52da73676c654?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/d6d1d054ded44c6b71c5ad207ee6ab9769ab199182963c14fbd52da73676c654?s=96&d=mm&r=g","caption":"lfdigital"},"url":"https:\/\/www.lf.com.tr\/tr\/author\/lfdigital\/"}]}},"_links":{"self":[{"href":"https:\/\/www.lf.com.tr\/tr\/wp-json\/wp\/v2\/posts\/3502","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.lf.com.tr\/tr\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.lf.com.tr\/tr\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.lf.com.tr\/tr\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.lf.com.tr\/tr\/wp-json\/wp\/v2\/comments?post=3502"}],"version-history":[{"count":1,"href":"https:\/\/www.lf.com.tr\/tr\/wp-json\/wp\/v2\/posts\/3502\/revisions"}],"predecessor-version":[{"id":8376,"href":"https:\/\/www.lf.com.tr\/tr\/wp-json\/wp\/v2\/posts\/3502\/revisions\/8376"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.lf.com.tr\/tr\/wp-json\/wp\/v2\/media\/4124"}],"wp:attachment":[{"href":"https:\/\/www.lf.com.tr\/tr\/wp-json\/wp\/v2\/media?parent=3502"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.lf.com.tr\/tr\/wp-json\/wp\/v2\/categories?post=3502"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.lf.com.tr\/tr\/wp-json\/wp\/v2\/tags?post=3502"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}