A lot of jobs within the IT field are somewhat invisible…what I mean is most of the company never really sees you. And one of the most invisible positions is that of a Network Engineer. Few people know we exist, and that’s ok by most of us, as we are usually very private by nature. Because of that, though, most people don’t really understand what we do and how important our jobs are.

When you have a few minutes, please read All Systems Down (pdf) by Scott Berinato for CIO magazine back in 2003. It chronicles a major network crisis which occurred in late 2002 at a large medical center in Boston. It is an incredible read!! In fact, I make sure I read it once a year. Why?

Because it reminds me of how important my job is. Sure, I don’t work at a hospital where lives are at stake, but still…the job I do is vitally important to over a thousand people every day. Their ability to get their jobs done in a quick and efficient manner relies largely on the stability and availability of my network. Plus, I learn a lot from this article in terms of attacking problems and working together. And I have to say a big “Thank you” to the CIO (John Halamka) which shared this story…the lessons learned is something every Network Engineer needs to take heed of.

Let me know what you think.

(Note: The article was not written by a Network Engineer, but a CIO journalist. As such, some of his descriptions are not 100% technically correct. But it does not in any way degrade the content and impact of the article.)

This post is part of a series that covers what I feel to be the main (or core) tasks that a Network Engineer is responsible for. See my Know Your Network – Introduction page for more details.

Documentation is a big subject, and can be overwhelming. So lets keep this simple. First things first…you need a document that shows all of your wide area network (WAN) circuits. This document should include the information needed to troubleshoot any issues, open tickets with the carriers when things do go wrong, and basically include all information needed to manage your network. Here is a very sanitized copy of my WAN drawing…(I removed about half of the drawing and dummied up the remaining confidential information)…

ACME_Network_Diagram (pdf)

Things to include for each circuit…

• Carrier circuit ID (also LEC ID when possible)
• Bandwidth
• IP addressing (both public and private)
• Location (remote site)
• LAN subnet(s) at remote site
• Router make/model
• Secure modem information (for those sites that have out-of-band management)

When creating the document, I would also recommend…

• Use Microsoft Visio. It is uniquely designed for this type of work, and has a number of templates and objects which will make the process easier. Also, most vendors (such as Cisco, HP, Juniper, etc) have created their own Visio objects of their hardware, and put them in files called “stencils” which you can download and add to Visio. This is VERY convenient! (Example: just Google “cisco stencils”)
• Use color!! By this, I mean do all IP addressing in red, circuit ID’s in blue, PRI circuits in black…etc. Choose your own colors, but stick with it…make it a standard.
• Create separate layers within the Visio drawing. One layer will be IP addressing, another layer Carrier circuit ID’s, and another circuit bandwidth….you get the idea. This does take time to initially setup, but once it is done, you will have a document that you can easily customize. As an example…you have a vendor meet and they request a network diagram. You don’t want to give them a drawing that shows any confidential information, so prior to printing or saving as a PDF, you can turn off certain layers, such as IP addressing and circuit ID’s. This makes sharing your network diagram both easy and secure, and you don’t have to maintain multiple documents.
• When creating host names for your network devices (routers, switches, etc) I would use a standard naming convention that has meaning. For example, look at the host names for the construction offices on my sample drawing…the first three letters are all “con” for construction. The next three letters show location (city), and then the final three characters show what the device is (rt-router, sw-switch, fw-firewall, ap-accesspoint, etc), followed by a number (1,2,3, etc) for each device, as there could be more than one at a location, such as switches.
• Include a Legend that shows all carrier contact information and drawing color definitions.
• Make it easy to read…print it out on standard tabloid paper (11″x17″). You can fold it in half for easy storage in your laptop case. Also, save as PDF and send to your smartphone and tablet.

This is one of the most important documents you will create and it will greatly aid you in managing your network. If you take your time and do it right, it will serve you well over the years, and will be easy to maintain and update.

Hope this helps…let me know what you think!

Finally. My life has calmed down (a bit anyway), and I’m able to get back to my website and do some posting.

I’m going to start a series of posts having to do with the key responsibilities of a network engineer. If you are a new network engineer and just starting out, what are the main tasks you should concentrate on? Or, perhaps you have been a network engineer for a while, but work is keeping you so busy that you are concerned about forgetting to do key tasks in managing the network. I also have seen some network engineers so busy playing with the latest cool toys, that they end up neglecting their main responsibility. Either way, what are the key responsibilities and/or tasks that need to be done to properly manage a network? Here are my key areas that I make sure and focus on…

1. Documentation – Know what networks you have (carriers, circuit id’s, support information, IP address assignments, etc.). Updated: WAN Drawing
2. Backups – Maintain proper backups of all your key network infrastructure (router configs and IOS images, switch configs, firewall configs and filter descriptions, along with backup/VMDK files of network related servers).
3. Logging – this includes SYSLOG’s from your network devices for user access tracking, alerting on device failures, configuration changes, power outages, etc.
4. Network Outages – Proactively monitor your network for any outages, and be ready to respond quickly and accurately. (It’s very cool to call a remote site letting them know of a network outage, and they haven’t even noticed it yet.)
5. Circuit Utilization – Know what traffic is running across your network, and be able to quickly identify applications that might be hogging too much bandwidth or be misbehaving. This will also give you the ability to perform capacity planning for future needs.
6. Perimeter Protection – For the most part, this covers your firewall and any perimeter router(s) you may have in place. Tighten down the security on these devices per best practices.
7. Cool Tools – Once you have the basics down, then you can start looking at some of the new tools and applications that can assist you in maintaining a robust and secure network (IE: Intrusion Protection Systems (IDS/IPS), Security Information and Event Management (SIEM’s), etc.)

Over the next several weeks I will take a more detailed look into each of the above items, and show you what I use to handle these tasks. Let me know what you think.

Thanks!