Get Website URL From Company Name in Python (Working Code)

The first script I ever wrote to get website URLs from company names died at name 23. Not name 2,000. Name 23. I was scraping Google, and Google noticed.

My IP got flagged, my results turned into empty strings, and the lead list I’d promised our sales team by Friday was maybe a third done. Fun week.

I run demand gen for a B2B team, and matching a raw company name to its real domain is step zero for almost everything we do, from data enrichment to routing inbound leads. So I rebuilt that broken scraper into a small Python pipeline that actually holds up. And in this guide, I’ll walk you through it step by step: clean the names, look them up, process a whole CSV, and go async when the list gets big.

Every code block here is runnable. Let’s get into it.

📌 Quick take: The pipeline is short: clean the company name → call a name-to-domain API → fall back to domain guessing for misses → cache every result → switch to async when your list passes 1,000 rows. That's the whole article in one line.

What You Need Before You Start

You need Python 3.9 or newer, one pip command, and a list of company names. That’s it.

Here’s the checklist:

• Python 3.9+ installed on your machine
• Three libraries: pip install requests pandas aiohttp
• A CSV file with a company_name column
• An API key for a name-to-domain lookup (a free tier is fine to start)

The requests library handles single lookups, pandas handles the CSV, and aiohttp comes in at the end for speed. Nothing exotic.

Three Ways to Get a Company Website URL in Python

You have three realistic routes: scrape a search engine, guess the domain, or query a name-to-domain API. I’ve shipped all three, and they are not created equal.

Route	Works for	Breaks when	Cost
Scraping Google	A handful of test names	Blocks hit after roughly two dozen requests	Free, plus your sanity
Domain guessing	Obvious names like “Shopify”	Rebrands, odd TLDs, non-US companies	Free
Name-to-domain API	Production lists at any size	You run out of credits	Low per lookup, free tier exists

So why not build the scraper here? Because it fails exactly when you need it. Search engines detect automated queries fast, and in my own runs the blocks started after about 23 requests. Delays and rotating user agents only postponed the ban.

🔍 Reality check: Before you point a bot at any site, read its rules. Google's own robots.txt and crawling guidance spells out what automated traffic is allowed. Most search scraping isn't.

This guide builds the API-first pipeline, with domain guessing as the free fallback. It’s the combination that survived production for me. Here’s exactly how to build it.

Step 1: Clean the Company Names First

Cleaning your input names is the highest-ROI step in this whole pipeline. And almost every tutorial skips it.

Real CRM exports don’t say “Acme”. They say “The Acme Corporation, Inc.” or “ACME Corp – US Branch”. Lookups choke on that noise. A little data cleansing up front lifts your match rate more than any clever code later.

Here’s a cleaner built on Python’s re module that strips legal suffixes, punctuation, and messy spacing:

import re

SUFFIX_RE = re.compile(
    r"\b(incorporated|corporation|company|limited|holdings|group"
    r"|inc|corp|llc|ltd|gmbh|plc|co)\b\.?",
    re.IGNORECASE,
)

def clean_name(name):
    name = name.lower().strip()
    name = SUFFIX_RE.sub("", name)
    name = re.sub(r"[^a-z0-9&\s-]", " ", name)
    return re.sub(r"\s+", " ", name).strip()

print(clean_name("The Acme Corporation, Inc."))  # the acme

Ten lines. But those ten lines fix the number one cause of failed lookups: the name in your file doesn’t match the name in anyone’s database.

Names are clean. Now let’s look one up.

Step 2: Look Up One Name With a Name-to-Domain API

A name-to-domain API checks your company name against a maintained database and returns the verified domain. No bot detection, no HTML parsing, no IP bans.

You send a POST request with the company name and a country code, and you get JSON back. Here’s the single-lookup function I use, built on the Company URL Finder endpoint (you can grab a free API key to follow along):

import requests

API_URL = "https://api.companyurlfinder.com/v1/services/name_to_domain"
HEADERS = {
    "x-api-key": "<your_api_key>",
    "Content-Type": "application/x-www-form-urlencoded",
}

def get_domain_api(company_name, country_code="US"):
    payload = {"company_name": company_name, "country_code": country_code}
    try:
        response = requests.post(API_URL, headers=HEADERS, data=payload, timeout=10)
        data = response.json()
        if data["status"] == 1 and data["data"]["exists"]:
            return data["data"]["domain"]
    except (requests.RequestException, ValueError, KeyError):
        pass
    return None

print(get_domain_api("Microsoft"))  # prints the matched domain

The response shape is simple: status tells you the call worked, data.exists tells you a match was found, and data.domain is the answer. The service does fuzzy data matching on the name, so “Acme, Inc.” and “ACME Incorporated” both land on the same domain.

Why pass the raw name instead of the cleaned one? Because the API handles its own normalization, and extra context can help it disambiguate. I keep the cleaned version for my cache key and my fallback. You’ll see both in Step 4.

And if you want a deeper dive into just this endpoint, I wrote a separate company name to domain API tutorial in Python that covers error handling and response codes in more detail.

Step 3: Add a Free Fallback With Domain Guessing

Guessing works like it sounds: build likely URLs straight from the cleaned name, then check which one answers. “Acme” becomes acme.com, acme.io, and so on.

import requests

def guess_domain(cleaned):
    slug = cleaned.replace(" ", "").replace("&", "and")
    candidates = [
        f"https://www.{slug}.com",
        f"https://{slug}.com",
        f"https://{slug}.io",
        f"https://{slug}.net",
    ]
    for url in candidates:
        try:
            r = requests.head(url, timeout=3, allow_redirects=True)
            if r.status_code == 200:
                return r.url
        except requests.RequestException:
            continue
    return None

A HEAD request fetches only the response headers, not the page, so each check is fast. Note that I return r.url, the final URL after redirects. Because a redirect from acmeinc.com to acme.com is actually a strong signal you’ve found the real brand domain.

Be honest with yourself about this method, though. In my runs it only resolved the obvious names, roughly the easy half of a typical B2B list. Rebrands, .ai domains, and non-US companies slip right through. It’s a free first pass, not a solution.

So far we can handle one name. But your list has hundreds. Let’s fix that.

Step 4: Run a Whole CSV Through the Pipeline With pandas

This is the step that turns a snippet into a tool: read the CSV, look up every name, write the results back out.

And there’s one trick that saves real money: a cache keyed on the cleaned name. Duplicate companies show up constantly in exported lists, and without a cache you pay for the same lookup twice.

import pandas as pd

df = pd.read_csv("companies.csv")  # needs a company_name column
cache = {}

def lookup(name):
    key = clean_name(name)
    if key in cache:
        return cache[key]
    domain = get_domain_api(name) or guess_domain(key)
    cache[key] = domain
    return domain

df["domain"] = df["company_name"].apply(lookup)
df.to_csv("companies_with_domains.csv", index=False)

Read that lookup function closely. It tries the API first because that’s where accuracy comes from, falls back to guessing for free, and stores whatever it finds. Even a None gets cached, so a known miss never burns a second credit.

💡 Pro tip: Key the cache on the CLEANED name, not the raw one. "Acme Inc." and "The Acme Corporation" collapse to the same key, which means one lookup instead of two. On my 2,000-row list, that alone cut the API calls by a noticeable chunk.

Not a Python person on your team? The same job works without code too. I covered that route in how to get website URLs from company names in bulk, which is the one I send to colleagues who live in spreadsheets.

Step 5: Go Async When the List Gets Big

So what happens when the list has 5,000 rows? Sequential requests spend most of their runtime waiting on the network, one response at a time.

asyncio plus aiohttp fires many requests at once and collects results as they land. This version reuses the API_URL and HEADERS from Step 2:

import asyncio
import aiohttp

async def fetch_domain(session, sem, name, country_code="US"):
    payload = {"company_name": name, "country_code": country_code}
    async with sem:
        try:
            async with session.post(API_URL, headers=HEADERS, data=payload) as resp:
                data = await resp.json()
                if data["status"] == 1 and data["data"]["exists"]:
                    return name, data["data"]["domain"]
        except aiohttp.ClientError:
            pass
        return name, None

async def run_batch(names, limit=10):
    sem = asyncio.Semaphore(limit)
    async with aiohttp.ClientSession() as session:
        tasks = [fetch_domain(session, sem, name) for name in names]
        return dict(await asyncio.gather(*tasks))

results = asyncio.run(run_batch(["Microsoft", "Shopify", "Zalando"]))
print(results)

The speedup is dramatic. My 2,000-name run went from most of an hour sequentially to a few minutes async. And if one request fails, the rest of the batch still finishes.

🧠 Remember: The Semaphore caps how many requests run at once (a semaphore is just a counter that makes extra tasks wait their turn). Keep it at 10 or so. Uncapped concurrency is how you turn a friendly API into a rate-limit wall.

What Breaks (and How to Fix It)

Every one of these failure modes has bitten me personally. Watch for them:

• Names that stay messy after cleaning: “Apex Solutions” could be twenty companies. Pass a country_code, or add industry context where your source has it.
• HTTP 429 errors: that’s the rate limit talking (429 means “too many requests”). Lower your semaphore, add a retry with a short sleep, and slow down.
• Directory traps: if you ever parse search results, the top hit is often a LinkedIn or Yelp page, not the company site. Filter known directory domains out.
• Parked domains: a guessed domain can return 200 and still be a for-sale page. Spot-check guesses before you trust them.
• Duplicate spend: no cache means paying twice for the same company. Step 4 already fixed this one for you.

The ambiguity problem is the deepest of these. When two candidates look close, string similarity scoring helps a lot, and I walk through it in my guide to fuzzy matching company names.

How to Verify Your Results

Never ship a list you haven’t measured. Two checks take five minutes.

First, print your match rate. Second, eyeball a random sample:

total = len(df)
found = df["domain"].notna().sum()
print(f"matched {found} of {total} ({found / total:.0%})")

sample = df.sample(min(20, total))
print(sample[["company_name", "domain"]])

If the sample looks clean and the match rate sits where you expect, ship it. If not, the misses usually cluster: one country, one weird naming pattern, one data source. Fix the cluster, not the whole script.

My 2,000-Name Test Run

Here’s the run that sold me on this pipeline. A couple of quarters back, sales handed me a conference export: about 2,000 raw company names across 18 countries, needed by Friday.

My old scraping instinct said rebuild the Google script. I resisted, barely. Instead I ran exactly the steps above: cleaned the names, sent them through the API async, and let guessing mop up a handful of stragglers.

The whole batch finished in roughly 12 minutes. Around 19 in 20 names came back with a verified domain, and international names with special characters resolved cleanly. But I won’t pretend it was perfect. A few dozen ambiguous names, mostly tiny local firms with generic words in their names, still needed a human to pick between candidates.

That’s the honest shape of this work. The script does 95% of the job in minutes. You still owe the last few rows your eyeballs.

How I Tested This (and Where It’s Limited)

Everything in this guide comes from running these exact scripts on my own lead lists, most recently in August 2026. The accuracy observations are mine, from my data, not from anyone’s marketing page.

So treat my numbers as directional. Your match rate depends on your input quality, your country mix, and how weird your company names are. Free tiers also cap monthly lookups, which matters at volume. Run the verification step on YOUR list before you trust any percentage, including mine.

Frequently Asked Questions

How do I get a business website URL?

Send the business name to a name-to-domain API, or guess likely domains and validate them. In Python, the API route is a single requests.post call that returns the verified domain, and it’s the reliable option for anything beyond a few names.

How to get a URL in Python?

Use the requests library: requests.get(url) fetches a page, and response.url gives you the final URL after redirects. For many URLs at once, aiohttp with asyncio fetches them concurrently.

How do I find a company’s URL?

Look the company name up in a verified name-to-domain database via an API. For a quick manual check, a search engine works fine. But for a list, automate it with the pipeline in this guide: clean, look up, fall back, verify.

How to find website URL from company name in bulk?

Load your names into pandas, run each through a cached lookup function, and write the results back to CSV. Go async once the list passes about 1,000 rows, and cap concurrency with a semaphore so you don’t hit rate limits.

How to get info from website in Python?

Fetch the page with requests, then parse the HTML with a library like BeautifulSoup. Check the site’s robots.txt and terms first, because not every site allows automated access, and an API is usually the cleaner path when one exists.

Is it legal to scrape Google search results in Python?

Scraping Google’s results violates its terms of service, and Google blocks automated queries quickly in practice. Laws vary by country, but the practical answer is the same either way: use an official API or a licensed data source instead.

It’s Time to Ship Your Script

Don’t spend a week on scraping infrastructure like I did. The pipeline is small and it works:

→ Clean the names → look up via API → guess as fallback → cache everything → go async at scale → verify a sample

Thirty minutes of Python, and your company names turn into real, verified website URLs. That’s a real plan, and it’ll hold up on a Friday deadline.

Want to skip the guessing entirely? Try the name-to-domain API free and run your own list through it. Then tell me in the comments how your match rate came out. You got this.