SEO and AI search

Technical SEO Basics: A Checklist You Can Run Without a Developer

By Sauravjit Singh · Updated 2026-11-05 · 6 min read

Short answer: Technical SEO basics are the settings that let search engines crawl, load and index your pages: a working robots.txt, a submitted XML sitemap, HTTPS, a mobile-friendly layout, reasonable page speed, correct canonical tags and no accidental noindex tags. On WordPress, Shopify or Wix, almost all of this is checkable from settings, with no code.

Most small Indian business sites don't fail on clever technical issues. They fail on simple ones: a staging setting left on, a plugin blocking the sitemap, a hero image that takes six seconds to load on a patchy mobile connection. This checklist covers what to check, where to find it on each platform, and what 'good enough' looks like.

It's part of our step-by-step SEO guide for beginners. Indexing errors you find in Search Console get their own page: fixing 'Crawled – currently not indexed'.

What does technical SEO actually cover?

Technical SEO covers everything that decides whether a search engine can reach, read and store your pages. If it's broken, the best content on the site never gets a chance to rank.

Think of it as three questions. Can crawlers get in (robots.txt, sitemap, server errors)? Can they read the page properly (mobile layout, speed, HTTPS)? Do they know which version to index (canonicals, redirects, noindex)?

The 12-point technical checklist

CheckWhere to lookGood enough looks like
Site is not set to hide from searchWordPress: Settings > Reading; Wix/Shopify: SEO or preferences settings'Discourage search engines' unticked; store password removed
robots.txt loadsyoursite.com/robots.txtLoads, doesn't say 'Disallow: /' for all bots
XML sitemap exists/sitemap.xml or /sitemap_index.xmlLists your real pages; submitted in Search Console
HTTPS everywhereOpen the http:// version of your URLRedirects to https:// with no browser warning
One preferred domainTry www and non-wwwOne redirects to the other
Mobile layoutYour own phone, plus Lighthouse in ChromeNo sideways scrolling, buttons easy to tap
Core Web VitalsPageSpeed Insights, Search ConsoleMostly 'Good' for mobile
No accidental noindexView page source, search 'noindex'Only on pages you deliberately hide
Canonical tagsView source, search 'canonical'Each page points to its own clean URL
Broken links and 404sSearch Console Pages report; a free crawlerNo important page returns 404
Redirects after URL changesOld URLs from any previous siteOld URLs 301-redirect to the right new page
AI crawlers allowed (if you want citations)robots.txtOAI-SearchBot, PerplexityBot and Bingbot not blocked

Practitioner tip: in our agency work, the single most common technical issue on freshly launched WordPress sites is the 'Discourage search engines' box still ticked from development. Check it before anything else.

Robots.txt and sitemap: what should they look like?

Robots.txt is a small text file that tells crawlers which parts of a site they may visit. A sitemap is a list of the URLs you want indexed. For a small business site, robots.txt should block almost nothing, and the sitemap should be generated automatically.

A sensible robots.txt for a WordPress site looks roughly like this (shown as plain lines): 'User-agent: *', 'Disallow: /wp-admin/', 'Allow: /wp-admin/admin-ajax.php', 'Sitemap: https://yoursite.com/sitemap_index.xml'. Shopify and Wix generate their own; you can view them but rarely need to change them.

Remember that robots.txt controls crawling, not indexing. Blocking a page in robots.txt doesn't reliably remove it from Google; a noindex tag does, and the page must be crawlable for Google to see that tag.

Should you allow AI crawlers like OAI-SearchBot and PerplexityBot?

If you want to be cited in ChatGPT search and Perplexity answers, generally yes. These crawlers fetch pages so the engines can show and link them in answers; blocking them makes citation less likely.

The names worth knowing, as of November 2026 (check each company's documentation, since names and purposes change):

  • OAI-SearchBot: OpenAI's crawler for ChatGPT search results. Allowing it is what matters for appearing as a cited source.
  • GPTBot: OpenAI's crawler for model training. You can block it separately if you don't want content used for training, without affecting OAI-SearchBot.
  • PerplexityBot: Perplexity's crawler for its search index.
  • Bingbot: Bing's crawler; Bing's index grounds Copilot, so don't block it.
  • Google-Extended: a robots.txt control for whether Google may use your content to train and ground its Gemini models. Google says it doesn't affect Google Search rankings or inclusion, and it isn't a separate crawler.

A worked example: say you run a Janakpuri coaching centre and want search citations but not training use. Your robots.txt might add 'User-agent: GPTBot' followed by 'Disallow: /', and leave everything else open. Also check your security plugin or CDN settings: some 'block AI bots' toggles block search crawlers too. To see whether these visits turn into traffic, set up tracking for AI referrals in GA4.

What are Core Web Vitals and how fast is fast enough?

Core Web Vitals are Google's three page-experience measurements: LCP (how quickly the main content appears; good is 2.5 seconds or less), INP (how quickly the page reacts to taps; good is 200 milliseconds or less) and CLS (how much the layout jumps around; good is 0.1 or less).

They're a ranking factor but a modest one. A slow page with the best answer can still outrank a fast thin one. Treat speed as a user problem first: a mobile visitor on a weak connection leaves a slow page.

  1. Run your home page and one product or service page through PageSpeed Insights and note the mobile scores.
  2. Compress large images. Many slow small-business sites have hero images of 2–5 MB; under 200 KB is usually achievable with WebP.
  3. Remove plugins and apps you don't use. Each one can add scripts to every page.
  4. On WordPress, add a caching plugin and lazy-load images below the fold.
  5. On Shopify, remove leftover app code after uninstalling apps; it often stays in the theme.
  6. Recheck after a week. Field data in Search Console updates slowly, over roughly 28 days.

Canonical tags and duplicate pages, in plain terms

A canonical tag tells search engines which URL is the main version when several URLs show the same content. It prevents your ranking signals being split across duplicates.

Common duplicates on Indian sites: Shopify products reachable through several collection URLs, WooCommerce filter and sort URLs, and UTM-tagged links shared on WhatsApp. Shopify and most WordPress SEO plugins add self-referencing canonicals automatically. Your job is to spot when they're wrong, for example every page pointing its canonical to the home page, which some badly coded themes do.

What should you check after a redesign or migration?

Most serious technical SEO damage on small sites happens during a redesign, a move to a new platform or a domain change. Rankings drop because old URLs stop working and nobody notices for weeks.

Before you switch anything over, export a list of every URL on the old site from a crawler or from Search Console's Performance report. That list is your safety net.

  1. Map every old URL to its closest new URL in a sheet, especially pages that get clicks in Search Console.
  2. Set up 301 redirects for every mapped URL before the new site goes live, not after.
  3. On launch day, untick any 'hide from search' setting and check robots.txt on the live domain.
  4. Submit the new sitemap in Search Console and Bing Webmaster Tools.
  5. Spot-check 20 old URLs in a browser to confirm each lands on the right new page.
  6. Watch the Pages report and Performance report daily for two weeks, then weekly for two months.

A practical threshold: if clicks fall by more than roughly a third in the first two weeks and don't recover, check redirects and indexing before blaming the design. In agency work, a missed redirect map is behind most post-launch drops we've been asked to investigate.

Platform notes: WordPress, Shopify and Wix

PlatformUsually handled for youWatch out for
WordPressSitemaps and canonicals via an SEO pluginSearch visibility box, plugin bloat, cheap hosting slowing pages
ShopifyHTTPS, sitemap, robots.txt, canonicalsApp leftovers, duplicate collection URLs, heavy themes
WixHTTPS, sitemap, mobile layoutHeavy animations, separate mobile edits drifting from desktop

Once the technical basics pass, move to content: our on-page SEO checklist is the next step. To check everything together once a quarter, use this DIY SEO audit. For official definitions, Google's Search Central documentation is the primary source. Technical SEO is taught with live sites in our advanced digital marketing course.

Frequently asked questions

Is technical SEO hard to learn?

The basics in this checklist are not; they're mostly settings and checks. Deeper areas like JavaScript rendering or crawl budgets on very large sites need more technical skill, but small business sites rarely need them.

Does blocking GPTBot hurt my Google rankings?

No. GPTBot is OpenAI's training crawler and has nothing to do with Google Search. Blocking it also doesn't block OAI-SearchBot, which is the crawler that matters for ChatGPT search citations.

How often should I check technical SEO?

Glance at Search Console's Pages and Core Web Vitals reports monthly, and run the full checklist after any redesign, theme change, migration or plugin overhaul. Those are when things usually break.

Do I need a fast hosting plan for SEO?

You need hosting that doesn't make pages slow or go down often. Very cheap shared plans can struggle once you add plugins; moving to a better plan or a managed host often fixes speed more than any plugin.

What is a 301 redirect?

A 301 redirect permanently sends visitors and search engines from an old URL to a new one. Use it whenever you change a page's URL so you don't lose the old page's links and rankings.

Want to learn this with a trainer?

Leave your details and our counsellor will call you within one working day with batch dates, fees and the right course for you.

  • Free counselling call, no obligation to join
  • Weekday and weekend batches, online or offline
  • Small batches of 15 to 20 students

Or chat on WhatsApp

Enter a 10-digit mobile number.

We use your details only to contact you about Spettro School courses.

Chat on WhatsApp