How To Bypass Dns, You don't have to wait for DNS propagation Options

[Ron Carnell]

I was on the phone the other day with a long-time forum friend and, when the conversation briefly touched on goDaddy, other registrars, and DNS propagation, she voiced a complaint I've heard many times over the years.

It can be frustrating not being able to reach a newly created or newly transferred domain until DNS kicks in for your particular location. If you don't know what DNS propagation is, run a Google Search for it and read almost any of the articles that come up in the SERP. The numbers vary from article to article, and indeed the numbers have greatly improved in recent years, but the advice across the board is pretty darn consistent. You just have to wait.

Actually, no . . . you don't have to wait.

My friend's complaint was common, but it nonetheless surprised me because we've been sharing forum space for nearly ten years and I know I've addressed this many times in the past. Somehow, though, she had missed all those posts and I promised her I'd again write up the instructions for Bypassing DNS. If you haven't figured it out yet, this is it.

Domain Name System

Quick review.

The foundation behind the Internet is the TCP/IP protocol and the foundation behind TCP/IP are IP addresses in the form of 70.87.92.158 (that's the IP for cre8asiteforums.com, btw). DNS is simply a distributed database, spread around the entire world, that overlays IP addresses with domain names. When you click on a link or type in an URL, your browser contacts a named server somewhere that accesses that database and returns the correct IP address. ONLY with that IP address can you actually get to a site.

When you register a new domain or transfer an old one, you have to tell your registrar where it can find the database entry for your domain. You do that by pointing the registrar to a named server, usually provided by your host, sometime by your registrar. That database entry for your domain is then essentially copied, over and over and over, to the distributed database around the world. Unfortunately, that's not instantaneous, so for some undetermined amount of time, parts of the distributed database will have old information and other parts will have new.

IP Addresses

If the company who is hosting your new or transferred domain gives you an IP address, you can usually access the site to upload files long before DNS propagates or, indeed, before the new site is even registered with your registrar. So, http://70.87.92.158/ will access the forums just as well as http://cre8asiteforums.com/ will. Usually, that's all you have and sometimes that's all you need.

It gets just a little more complicated if your new or transferred domain doesn't have its own dedicated IP address. Available IP addresses are finite and limited, while domain names are not, so in today's world it's very common (and responsible) for a domain to share it's IP address with many other domains. When a browser goes to that IP address it has to send a HOST header to tell the web server which domain it wants. On many, many web servers, certainly all those running Apache, you can fudge the HOST header by sending a tilde user site. So, http://70.87.92.158/~cre8/ would get you to the forums, too, even we were sharing that IP address with other domains. (Note that the user name isn't necessarily the same as the domain name. Indeed, for security purposes it's best if they're different. Your host, obviously, will provide the correct user name.)

Sometimes, unfortunately, the IP address isn't enough.

You can usually upload all your files with just the IP address, but testing and configuring the site can still be a big problem. If your internal links are absolute, for example, instead of relative, any click to a new page will take you away from the IP address and back to the domain. It can also be extremely difficult to configure dynamic software (like these forums) without a domain name that actually works. Fortunately, there are solutions.

DNS Servers a.k.a. named servers

There are hundreds of thousands of DNS servers that comprise the distributed database that is the Domain Name System. When you click on a link, your browser goes to one of those DNS servers to start the search that will culminate in a usable IP address. That particular DNS server, for you, is typically defined by your ISP.

But it doesn't have to be!

The problem with using your ISP's DNS server is that you have no control over when it updates its local database. You go to your registrar to tell them you are transferring a domain, you give the registrar the IP addresses of two named servers that have the database entry for your domain, and then you wait for your local ISP to go get that information from your registrar? I don't think so.

You already have the IP addresses of two DNS servers that you KNOW has the updated database record for your domain, right? Use those instead of your ISP's DNS servers and you'll gain instant access to your domain. This will work even BEFORE you tell your registrar you've officially moved the domain.

I'm going to give you instructions for setting your own DNS servers for XP, but I know it was similar for Win98 and I suspect it won't be greatly different for Vista. I don't know how to do it for a Mac, but I have no doubt it can be done.

For XP, go to Start -> Control Panel -> Network Connections and then slide your mouse over to click on one of your ISP connections. I have several, you might have only one (or possibly none; see below). Click on the Properties button, then click on the Networking tab. In the area labeled "This connection uses the following items:" make sure that "Internet Protocol (TCP/IP" is highlighted and then click on the Properties button.

This is what you should see:



On your computer, you'll probably find that "Obtain DNS server address automatically" is enabled. That's your ISP's named server. Click on "Use the following DNS server addresses" to enable it, and then enter the IP address of the two named servers your host gave you for your domain. Click on OK and keep clicking OK until you get out of the Control Panel dialogs.

If you are already connected to the Internet with that particular connection, you'll probably see a warning that the change won't go into effect until you reconnect. In any case, you need to disconnect from the Internet and reconnect using THAT connection. To test if your change is working, jump out to a Command Prompt, clear your cache, then ping your domain name. I've highlighted the commands below. Your computer should now return the correct IP address.



We used the Command Prompt to test the change because your browser won't always be so quick to accommodate. Browsers cache stuff, remember? You'll probably have to clear your browser cache and maybe close it, too, before it will reconnect to the new DNS server. Firefox can be particularly aggressive in keeping it's cache. Worse case, you'll have to clear the cache and reboot your computer, but sooner or later, your browser, FTP client, etcetera will all be able to access your domain -- even if no one else in the world can.

Window's Control Panel can be used as detailed to change DNS Servers for any network connections it controls. I have two ISPs, my satellite broadband and my 11-year-old dial-up account for those frustrating times when the satellite isn't working. Windows even lets you set up multiple connections for the same ISP, so I have several for my dial-up to reach different local numbers. Every one of those connections in the Control Panel can be configured to use a different DNS server, so this is a very flexible way to configure your machine. When we moved the forums to a new server, I had one connection that would reach cre8asiteforums.com on the old server and a different connection that would reach it on the new server. I just changed connections at will. I like that flexibility.

However, my satellite broadband is an "always on" connection and, unlike my dial-up, doesn't have a connection setting in the Control Panel. I have to set the DNS server for the satellite in proprietary screens, and you might have to do the same. Obviously, I can't tell you how to do that, so you'll have to ask your ISP. (On some broadbands, using the "Local Area Connection" in your Control Panel will also control the broadband connection. Try it, it won't hurt, and you can always set it back.)

The HOSTS file

There's always a cost to flexibility, and wending through the Window's Control Panel can sometimes feel a bit daunting. There is an easier, albeit somewhat less flexible way, to bypass the Domain Named System.

Windows keeps close track of a file called hosts (usually found at C:\windows\system32\drivers\etc\hosts though you can Search for it on your computer) that it always consults before it EVER tries to contact a DNS server. Here's what the hosts file looks like in my editor:



You'll likely find localhost already defined in the file, and you can see in the image above that I've added an entry for the forums (be sure to add the www version, too). Call this file up in your favorite text editor (like Notepad, but NOT a word processor) and add a similar entry for your new or transferred domain. Save it, close it, and within seconds Windows will start using that IP address instead of requesting one from your DNS server. As before, you should test the change by pinging the domain in a Command Prompt window, because your browsers will have to be forced to relinquish their cache.

The hosts file affects ALL of your Internet connections, making it slightly less flexible but often a whole lot easier. If you're jumping back and forth between an old server and a new server, you might find yourself editing this file a lot. I keep a shortcut to it on my desktop just to make it easier.

(If you've already moved to Vista, you might find your own security keeps you from editing the hosts file. See the work-around at Microsoft's site.)

Summary

Which method you use to bypass DNS is probably going to be a personal choice. I like the Control Panel because it's more flexible, but keeping track of which connections point where can sometimes get confusing (and it's caused me to inadvertently do the right thing on the wrong machine before, too).

The hosts file can be an incredibly powerful tool even beyond bypassing DNS, but you want to be careful it doesn't grow overly large (it'll slow down your computer).

Either method can free you from waiting for DNS propagation.

[rynert]

Good information Ron - for a short while I thought you were going to come up with something I thought was not possible, and by-pass the propogation delay for everybody - not just myself!

I was reading, waiting for that magic bit of info that I had somehow not found out all those years.. then the penny dropped! lol

[eKstreme]

You can bypass the delay by setting the DNS refresh time to be something very short, like an hour or so. Set it up a few days before the move, do the move, and reset it back to the normal value a few days later.

[DianeV]

Hi, Ron, and thanks.

I'd known about the Hosts file (that's what we were using to access the site at the new host) -- but not about the internal DNS settings. Much appreciated, and what a reference this is!

[Tim]

Excellent post Ron, thanks!

I'd never thought of changing the DNS servers to the authoritative ones before.

On many, many web servers, certainly all those running Apache, you can fudge the HOST header by sending a tilde user site.

Are you sure it's all Apache servers? I've only seen it working before on cPanel based servers - and thinking about it, virtual hosts aren't necessarily based on a username, so I don't think it'd work. cPanel certainly has it built in though.

You can bypass the delay by setting the DNS refresh time to be something very short, like an hour or so. Set it up a few days before the move, do the move, and reset it back to the normal value a few days later.

That's a good idea too - it's the TTL setting you'd need to change. Setting it to something like 5 minutes rather than 4 hours or 24 hours would be useful. However, some ISPs will ignore TTL and just cache the old DNS records for as long as they feel like it. So it won't always be a foolproof solution.

[Ron Carnell]

Are you sure it's all Apache servers? I've only seen it working before on cPanel based servers ...

You could certainly be right, Tim. The tilde access was fairly standard on Unix based machines back in the day (long before CPanel came on the scene), but I'll admit I haven't tried to use it in something like ten years or more. Apache might have gone through one or two revisions since then.

That's a good idea too - it's the TTL setting you'd need to change.

TTL, as in Time To Live. There are actually two TTL setting, I believe, in the DNS records (not to be confused with a setting of the same name for routers). The other is the "negative" TTL which tells named servers how long to cache a negative response (as in, that domain ain't here). You don't necessarily need to change that one, but you do need to recognize the difference. And for a brand new domain, the negative TTL in the SOA record will determine how often other named servers will return to query the new domain name. An unusually high negative TTL can keep a brand new domain hidden for quite a while, though the effect is typically localized and not widespread.

If you have that kind of access to the DNS records, however, there's an even better way.

I think the best procedure, when you have access to the DNS records, is to change the TTL several days before the move is going to be made, perhaps as much as a week, depending on your current setting. If your TTL is set to 86400, for example, which is one day, then you should change your TTL at least 86,401 seconds before you want to move the domain. That insures that all the named servers have flushed their cache and will now abide by the much shorter TTL you've set (five or ten minutes is usually good).

When the move has been made, go back to the DNS record and change all of the IP addresses to point to the new server. With those changed, you don't even have to wait for the root servers to be updated. Instant access.

However, some ISPs will ignore TTL and just cache the old DNS records for as long as they feel like it.

Honestly, Tim, in my opinion that's an Urban Myth that has never been established. There is no setting in any of the DNS server software I've seen to easily accommodate ignoring TTL parameters. For an ISP to do that, they would have to customize bind, which is hardly a trivial task.

The ones who WILL ignore TTL are the search engine spiders. They reportedly cache the IP address of a site so they don't have to keep querying named servers (every second counts when you're crawling the whole Web) and there's never been any indication they pay the least bit of attention to TTL. Used to be it could take a solid month to get them to flush their cache, which is why I've always maintained copies of both the old and new sites for at least that long. These days, they're much, much better and it's been years since I heard of a site getting dropped because the spider went to the old server and found it gone. Color me paranoid, but I nonetheless still maintain two copies until I see all the spiders crawling the new site. It's pretty cheap insurance.