Technical SEO Basics: A Checklist You Can Run Without a Developer
Short answer: Technical SEO basics are the settings that let search engines crawl, load and index your pages: a working robots.txt, a submitted XML sitemap, HTTPS, a mobile-friendly layout, reasonable page speed, correct canonical tags and no accidental noindex tags. On WordPress, Shopify or Wix, almost all of this is checkable from settings, with no code.
Most small Indian business sites don't fail on clever technical issues. They fail on simple ones: a staging setting left on, a plugin blocking the sitemap, a hero image that takes six seconds to load on a patchy mobile connection. This checklist covers what to check, where to find it on each platform, and what 'good enough' looks like.
It's part of our step-by-step SEO guide for beginners. Indexing errors you find in Search Console get their own page: fixing 'Crawled – currently not indexed'.
What does technical SEO actually cover?
Technical SEO covers everything that decides whether a search engine can reach, read and store your pages. If it's broken, the best content on the site never gets a chance to rank.
Think of it as three questions. Can crawlers get in (robots.txt, sitemap, server errors)? Can they read the page properly (mobile layout, speed, HTTPS)? Do they know which version to index (canonicals, redirects, noindex)?
The 12-point technical checklist
| Check | Where to look | Good enough looks like |
|---|---|---|
| Site is not set to hide from search | WordPress: Settings > Reading; Wix/Shopify: SEO or preferences settings | 'Discourage search engines' unticked; store password removed |
| robots.txt loads | yoursite.com/robots.txt | Loads, doesn't say 'Disallow: /' for all bots |
| XML sitemap exists | /sitemap.xml or /sitemap_index.xml | Lists your real pages; submitted in Search Console |
| HTTPS everywhere | Open the http:// version of your URL | Redirects to https:// with no browser warning |
| One preferred domain | Try www and non-www | One redirects to the other |
| Mobile layout | Your own phone, plus Lighthouse in Chrome | No sideways scrolling, buttons easy to tap |
| Core Web Vitals | PageSpeed Insights, Search Console | Mostly 'Good' for mobile |
| No accidental noindex | View page source, search 'noindex' | Only on pages you deliberately hide |
| Canonical tags | View source, search 'canonical' | Each page points to its own clean URL |
| Broken links and 404s | Search Console Pages report; a free crawler | No important page returns 404 |
| Redirects after URL changes | Old URLs from any previous site | Old URLs 301-redirect to the right new page |
| AI crawlers allowed (if you want citations) | robots.txt | OAI-SearchBot, PerplexityBot and Bingbot not blocked |
Practitioner tip: in our agency work, the single most common technical issue on freshly launched WordPress sites is the 'Discourage search engines' box still ticked from development. Check it before anything else.
Robots.txt and sitemap: what should they look like?
Robots.txt is a small text file that tells crawlers which parts of a site they may visit. A sitemap is a list of the URLs you want indexed. For a small business site, robots.txt should block almost nothing, and the sitemap should be generated automatically.
A sensible robots.txt for a WordPress site looks roughly like this (shown as plain lines): 'User-agent: *', 'Disallow: /wp-admin/', 'Allow: /wp-admin/admin-ajax.php', 'Sitemap: https://yoursite.com/sitemap_index.xml'. Shopify and Wix generate their own; you can view them but rarely need to change them.
Remember that robots.txt controls crawling, not indexing. Blocking a page in robots.txt doesn't reliably remove it from Google; a noindex tag does, and the page must be crawlable for Google to see that tag.
Should you allow AI crawlers like OAI-SearchBot and PerplexityBot?
If you want to be cited in ChatGPT search and Perplexity answers, generally yes. These crawlers fetch pages so the engines can show and link them in answers; blocking them makes citation less likely.
The names worth knowing, as of November 2026 (check each company's documentation, since names and purposes change):
- OAI-SearchBot: OpenAI's crawler for ChatGPT search results. Allowing it is what matters for appearing as a cited source.
- GPTBot: OpenAI's crawler for model training. You can block it separately if you don't want content used for training, without affecting OAI-SearchBot.
- PerplexityBot: Perplexity's crawler for its search index.
- Bingbot: Bing's crawler; Bing's index grounds Copilot, so don't block it.
- Google-Extended: a robots.txt control for whether Google may use your content to train and ground its Gemini models. Google says it doesn't affect Google Search rankings or inclusion, and it isn't a separate crawler.
A worked example: say you run a Janakpuri coaching centre and want search citations but not training use. Your robots.txt might add 'User-agent: GPTBot' followed by 'Disallow: /', and leave everything else open. Also check your security plugin or CDN settings: some 'block AI bots' toggles block search crawlers too. To see whether these visits turn into traffic, set up tracking for AI referrals in GA4.
What are Core Web Vitals and how fast is fast enough?
Core Web Vitals are Google's three page-experience measurements: LCP (how quickly the main content appears; good is 2.5 seconds or less), INP (how quickly the page reacts to taps; good is 200 milliseconds or less) and CLS (how much the layout jumps around; good is 0.1 or less).
They're a ranking factor but a modest one. A slow page with the best answer can still outrank a fast thin one. Treat speed as a user problem first: a mobile visitor on a weak connection leaves a slow page.
- Run your home page and one product or service page through PageSpeed Insights and note the mobile scores.
- Compress large images. Many slow small-business sites have hero images of 2–5 MB; under 200 KB is usually achievable with WebP.
- Remove plugins and apps you don't use. Each one can add scripts to every page.
- On WordPress, add a caching plugin and lazy-load images below the fold.
- On Shopify, remove leftover app code after uninstalling apps; it often stays in the theme.
- Recheck after a week. Field data in Search Console updates slowly, over roughly 28 days.
Canonical tags and duplicate pages, in plain terms
A canonical tag tells search engines which URL is the main version when several URLs show the same content. It prevents your ranking signals being split across duplicates.
Common duplicates on Indian sites: Shopify products reachable through several collection URLs, WooCommerce filter and sort URLs, and UTM-tagged links shared on WhatsApp. Shopify and most WordPress SEO plugins add self-referencing canonicals automatically. Your job is to spot when they're wrong, for example every page pointing its canonical to the home page, which some badly coded themes do.
What should you check after a redesign or migration?
Most serious technical SEO damage on small sites happens during a redesign, a move to a new platform or a domain change. Rankings drop because old URLs stop working and nobody notices for weeks.
Before you switch anything over, export a list of every URL on the old site from a crawler or from Search Console's Performance report. That list is your safety net.
- Map every old URL to its closest new URL in a sheet, especially pages that get clicks in Search Console.
- Set up 301 redirects for every mapped URL before the new site goes live, not after.
- On launch day, untick any 'hide from search' setting and check robots.txt on the live domain.
- Submit the new sitemap in Search Console and Bing Webmaster Tools.
- Spot-check 20 old URLs in a browser to confirm each lands on the right new page.
- Watch the Pages report and Performance report daily for two weeks, then weekly for two months.
A practical threshold: if clicks fall by more than roughly a third in the first two weeks and don't recover, check redirects and indexing before blaming the design. In agency work, a missed redirect map is behind most post-launch drops we've been asked to investigate.
Platform notes: WordPress, Shopify and Wix
| Platform | Usually handled for you | Watch out for |
|---|---|---|
| WordPress | Sitemaps and canonicals via an SEO plugin | Search visibility box, plugin bloat, cheap hosting slowing pages |
| Shopify | HTTPS, sitemap, robots.txt, canonicals | App leftovers, duplicate collection URLs, heavy themes |
| Wix | HTTPS, sitemap, mobile layout | Heavy animations, separate mobile edits drifting from desktop |
Once the technical basics pass, move to content: our on-page SEO checklist is the next step. To check everything together once a quarter, use this DIY SEO audit. For official definitions, Google's Search Central documentation is the primary source. Technical SEO is taught with live sites in our advanced digital marketing course.
Frequently asked questions
Is technical SEO hard to learn?
The basics in this checklist are not; they're mostly settings and checks. Deeper areas like JavaScript rendering or crawl budgets on very large sites need more technical skill, but small business sites rarely need them.
Does blocking GPTBot hurt my Google rankings?
No. GPTBot is OpenAI's training crawler and has nothing to do with Google Search. Blocking it also doesn't block OAI-SearchBot, which is the crawler that matters for ChatGPT search citations.
How often should I check technical SEO?
Glance at Search Console's Pages and Core Web Vitals reports monthly, and run the full checklist after any redesign, theme change, migration or plugin overhaul. Those are when things usually break.
Do I need a fast hosting plan for SEO?
You need hosting that doesn't make pages slow or go down often. Very cheap shared plans can struggle once you add plugins; moving to a better plan or a managed host often fixes speed more than any plugin.
What is a 301 redirect?
A 301 redirect permanently sends visitors and search engines from an old URL to a new one. Use it whenever you change a page's URL so you don't lose the old page's links and rankings.
Keep reading in SEO and AI search
- SEO for Beginners: A Step-by-Step Plan for a Real Small-Business Site Start here
- Rank Math vs Yoast: Which WordPress SEO Plugin Should Beginners Use?
- Google Business Profile Suspended: How to Fix It and Pass Video Verification
- Link Building for a New Website: How to Earn Your First Real Backlinks in India
- What Is AEO and GEO, and How Are They Different From SEO?
Courses: Advanced Digital Marketing & Performance Ads · AI-Powered Digital Marketing · All guides in SEO and AI search