Category Archives: Protocols

Office Phones Down on Monday Morning – DHCP Issue

If you are a Network Engineer, then having your phone ring early on a Monday morning is never a good thing. And that’s what happened this morning. Seems that none of our Cisco phones were working at the Corporate office…yep, none, as in 150 employees. (And it always seems to be the big offices…why does nothing go down at a little remote office that only has 3 employees?)

After talking to a couple of users (via cell phone of course), I realized that all of the phones were trying to get an IP address but were unable to.  (Phones were displaying:  Configuring IP). Well that is very interesting…and I recalled that the DHCP server was just replaced this past Friday. (Grrrrrr….how do you mess up a DHCP server?)

Well, it’s easy to mess things up if you don’t use the same IP address on the new server as the old server. Remember, DHCP is a layer two broadcast mechanism…the device booting up sends out a DHCP Request packet destined for all F’s, and the server responds.  Now, if the server is NOT on the same local network, then you need an IP helper statement such as this to properly forward the DHCP request…

interface Vlan11
description VoIP VLAN for Corporate office
ip address 10.11.2.1 255.255.255.0
ip helper-address 10.10.2.10

As you can see the switch will take any DHCP requests and forward as a unicast packet to the DHCP server. However, the new DHCP server had a different IP address. Why? I don’t know…however the server guy did fix the issue quickly, which was much appreciated. He simply changed the IP address of the new server to that of the old server…and boom, all of the phones started registering. You may be wondering about all of the PC’s on the network…they were working just fine as they were on the same local network as the DHCP server.

I then was able to finish shaving and get ready for the day. You just have to love Mondays…

Installing DIG on a Windows Machine

First up…a quick apology. Back on April 10th, I posted Verifying Proper Email Routing – MX Records. At the end of the post I mentioned I would talk about the trailing dot shown in the DNS records in my next post. Well, I basically forgot. Life just got too busy.

But before I do that, you really need to get DIG installed on your Windows PC (if you have Linux then you should already have DIG). So today, lets get DIG installed and tested…this is a great tool to have, and you will be surprised at how often you will use it.

The package we are going to install is BIND (Berkeley Internet Name Domain) from ISC (Internet Systems Consortium). ISC has been around for years, and most DNS servers on the Internet run BIND (whether Windows or Linux/UNIX based). Do the following…

  • Go to https://www.isc.org/downloads/bind
  • There is a lot of good info on this page concerning BIND and DNS…I would recommend reading this when you have a chance.
  • Click on Download BIND button
  • Expand the BIND menu
  • Click on the Download button to the right of the GREEN “Current-Stable” status
  • In the window that pops open, choose the correct version for your system…(click the appropriate blue box)
  • Next, create a directory in your C: drive called “bind”, move the BIND ZIP file to that directory, then double-click on the ZIP file. Extract all of the files in the \bind directory.

Now, lets test. Open up a command prompt and change to the BIND directory, then type in “dig” and press enter.  You should see something like this…(this example is showing DIG returning a list of the ROOT servers on the Internet)…

Running DIG at the command prompt

Running DIG at the command prompt

If it doesn’t work, there are a couple of common errors which are easily fixed…

32 bit installation error:  You may get an error stating “The application has failed to start because its side-by-side configuration is incorrect”.  If that’s the case, run the file “vcredist_x86.exe” which is located in the same “bind” directory. Then try running “dig” again…it should work.

64 bit installation error:  If you get the error message stating that MSVCR110.DLL is missing, then do a search on the C: drive for that file, and copy it to the “bind” directory. Running “dig” should now work.

Now you can play with DIG…some examples…

Some DIG examples

Some DIG examples

DIG is very powerful, and will quickly become your go-to tool when you have any DNS issues. I will talk more about using DIG in a future post.

Using Whois to Find Domain Ownership

Over the last several days, there has been a bit of a media skirmish concerning a report from the Global Energy Balance Network, a non-profit science group dedicated to preventative education to reduce obesity. A recent report of theirs stated that lack of exercise is primarily responsible for the dramatic upswing in the obesity rate here in the US, and not necessarily what we eat (such as sugary drinks). All well and good, and I would tend to agree with them…we have become a nation that sits on it’s butt.

However, news then surfaced that the report was funded in part by Coca-Cola Company. Hmmmm…that could tend to tarnish the report a bit. I heard that the domain name for Global Energy Balance Network (gebn.org) was registered to Coca-Cola. So last night (Tuesday 8/11), before I went to bed, I did a “whois” lookup on my Linux system, and sure enough…the domain was registered to Coca-Cola. Very interesting.

So, at work today (Wednesday 8/12) I ran another whois so i could screen capture it and put it in my blog as an example. Well guess what…the registration had changed. I was surprised. Here is what I found today…

Current WHOIS for GEBN.ORG

Current WHOIS for GEBN.ORG

As you can see, registration was updated this morning around 14:52 UTC (around 10:52 AM EDT). So, why the University of South Carolina? I’m guessing there is a relationship between the non-profit and the University…which is fairly common these days. Also, if you go to the website and check out the “About” page, there is a disclaimer stating that part of their funding is from the Coca-Cola Company…so they are not trying to hide anything.

Now, I’m just pointing this out as an example of domain name registration and some of the gotcha’s to be aware of…and for the use of the “whois” command, which is part of Linux.

Disclaimer:  Yep, I guess I need to fess up a bit too…being a good ol’ Georgia boy, I do love drinking my Coke and Dr. Pepper. I have a joke I tell friends that when I go to the doctor’s office and give blood, it fizzes.

CiscoLive and a Leap Second

Greetings everyone…

Yes, I’ve been way too busy…sorry for the lack of posts. I am going to try and get back in the swing of things!!

First, a bit of fun news…I’ll be out of town next week, down in San Diego, attending CiscoLive 2015!! And even better, my wife is going with me! I bought her a Social Events Pass…this allows her to attend all of the Keynote speeches, the evening vendor meets, and the closing night concert with Aerosmith at Petco Park (the Padre’s stadium)!! We are both very excited!! I will try to post some notes next week from CiscoLive…if I can find the time.

Next…since we are talking about time….on Tuesday, June 30th, there is going to be a leap second event. (I’m not kidding folks…this is actually very cool stuff!!) Over time, the Earth is gradually slowing down…and to keep the “real” time sync’d with our perception of time (sundown, etc.) they have to add a second to the day. The last time this happened was back in 2012. Is this something you need to worry about? Probably not, but you should be aware of it, and keep an eye on things that might be acting strange (frozen, pegged CPU, etc). Here is Cisco’s take on this years leap second.

Details:  The leap second is going to occur at 23:59:59 UTC on June 30th, 2015. In other words, one second before midnight UTC time. For me, here on the west coast with daylight savings time, it will be 4:59:59 PM in the afternoon (PDT). That final minute will actually last 61 seconds. This will be orchestrated by all of the NTP servers on the Internet…so if all of your network equipment is time sync’d to NTP servers, you should be fine. If you equipment is not time sync’d, well…I guess you don’t really care about all of this anyway. (And you are not being a good network engineer either…get your network time sync’d!!)

Linux and UNIX systems should handle this well…NTP will announce to any system running true NTP that a leap second will occur (there is a leap second flag that gets set). Windows systems do not know how to handle the NTP announcement, so they will just end up a second off. However, within 20 minutes they will be re-sync’d properly. What to watch for? Anything acting strange that relies on exact time…applications that use GPS would be key to watch.

So…you have been warned. Hopefully the addition of a leap second will not trigger Armageddon.

Verifying Proper Email Routing – MX Records

I had an issue come up today in which I needed to verify what mail servers were handling email for a particular domain. (I like having easy problems on a Friday!)

So, how do you answer this question? Simple…you need to look up the MX records associated with the domain in question. MX stands for Mail Exchange…which are DNS records of mail servers sitting on the Internet which handle email for a domain. Quick example using my handy DIG utility (available within Linux, or you can download it from isc.org for Windows)…

MX results for cisco.com

MX results for cisco.com

As you can see, the DNS reply gave us 3 MX records for mail servers that handle email for the domain “cisco.com”…

10 alln-mx-01.cisco.com.
30 aer-mx-01.cisco.com.
20 rcdn-mx-01.cisco.com.

The numbers in front of each line are known as “preference numbers” and establish which order the servers are to be used, with the smaller number being more preferred. In this case, mail servers will attempt to contact server alln-mx-01.cisco.com first, and if not successful, will then attempt server rcdn-mx-01.cisco.com….and so on. (And no, Cisco does not have a single server that takes care of all their email…most likely alln-mx-01 simply points to a large server cluster). A common technique you will see is to list several servers all with the same preference number…this allows for load-balancing among the servers (a bit crude, but it does work). HP handles load-balancing a bit differently…

MX results for hp.com

MX results for hp.com

I like HP’s solution…simple and efficient…there is only one MX record, but multiple “A” (Address) records that smtp.hp.com resolves to. How about Apple?….

MX results for apple.com

MX results for apple.com

As you can see, Apple handles load-balancing in a bit more complex manner, but it works very well…(sounds just like Apple, doesn’t it?). There are 5 preference number “10” servers and 5 “20” servers, and I bet they are spread out all over the place…different data centers in America and perhaps other parts of the world. Notice the single “100” preference server, which will only get used if none of the other servers are up and running. Knowing Apple, I’m sure this server is kept up to date and patched. But smaller organizations tend to setup a high number preference server as a last backup, which hardly ever gets used, and they tend to forget about it…maybe not keep up with patches and security updates. As a result, you will tend to see hackers go after the high numbered preference mail servers, as they may be an easier target.

Bonus question: Did you notice that all the host names and domain names ended with a “dot”, such as apple.com. and smtp.hp.com.? Know why? I’ll tell you in my next blog.

Hope this info was helpful…

The OSI Seven Layer Model – Why Learn It?

The number one reason I started this blog was to provide help and encouragement to new and prospective Network Engineers. Most of the time, I’ll be passing on experiences that I have in the field of networking. However, as I have time, I also want to post articles that I hope will take your understanding of networks up to the next level. I will include practical examples and exercises you can do at home which will give you a better understanding of networks, and I hope will make you a better Network Engineer. Today is my first post of this type. Please let me know if this is helpful (or not). Here goes…

The OSI Seven Layer Model. (I can you see wincing!!) Why bother, right? Most people I know learned just enough about the OSI model to answer the questions on some written test, and most likely have not given it a thought since. It’s just a bunch of theory, and doesn’t really help in the real world. Well, if that’s what you think, then you are missing a powerful tool in understanding networks AND in troubleshooting them. The best network engineers know the OSI model. And you can too.

Continue reading →

Cool Network Tools – digHD

A fellow co-worker showed me this iPad app for “dig” called digHD, and I have to say it’s great. I downloaded it immediately and started playing with it…VERY cool and helpful. Of course I prefer using DIG at the commandline in Linux or Windows, but this will do when I don’t have commandline access.  You can do a lot of DNS troubleshooting with this, and it’s both flexible and easy to use.  It does cost $2.99, but well worth it.  Here is a screenshot…

DIG'ing google.com

DIG’ing google.com

Enjoy!!  (And yes, I do need to get my iPad plugged in quickly!)

What Is – DHCP

DHCP – Dynamic Host Control Protocol

This is a wonderful and time saving little protocol. You already know that every device on a network has to have an IP address…but…how do you configure all of those devices and their IP address? Well, you could do it manually. Once. Twice. Maybe several times. But you would quickly realize the manual way is a pain in the butt….there has to be an easier way! Well, there is. Enter DHCP.

When you turn on most devices, as they boot up, they send out a packet onto the network basically saying “Help…I’m booting up and I need an IP address. Can anyone help me?” And if you have a server (or other device) running a DHCP process, then the answer is yes. The DHCP server will reply with an IP address, and several other bits of needed information, and just like that…the device has an IP address and can start communicating on the network.
Isn’t that cool? I think it is. I will post more about DCHP in the near future and we will see exactly what goes on during a DHCP request.