Only one more day before Halloween. We have a few horrifying IT tales to share just in time for All Hallows’ Eve.
[credit provider=”Channel 4, The IT Crowd” url=”http://www.channel4.com/”]
In honour of Halloween 2015, SolarWinds asked its community of IT professionals to share their creepiest IT stories. While they don’t involve eerie apparitions or dreadful demons, they do detail scenarios that would send chills down the spine of many IT professionals. Here are four stories to feast your eyes on.
#1 A rampant virus
Sleep picture from Shutterstock
George – Data Systems Manager: “A few years back I was working for a large financial institution, managing a systems and networking group. It was 7:20am and as I walk into my office I overhear several of our system security people talking in an animated fashion about something.
I went over to say good morning and I hear that they are talking about a virus that appears to be ping-ponging all over the building and sending emails to all of our contacts externally. It had been active for about 30 minutes or so. They told me that it was being spread by an email that was supposedly ‘penned’ by a senior person at the company. Their discussion was all about the virus and the type it was and why the virus software had not caught it and so on and so forth.
So I asked: ‘So we are in the process of taking the email servers off the network and isolating the problem and stopping the threat from spreading right?’ The sound of crickets permeated the area. I picked up my Nextel and called one of my guys. Three minutes later the issue was contained. The result – (300 PCs affected out of 2800).
Five hours later everything was back to normal. I related this to their manager a bit later in the day and his face had that look of horror that only a classic OMG moment can give. Needless to say there were a lot of sad faces about 45 minutes later after his team meeting.
Never let common sense get in the way of a good discussion!”
#2 When “ghoul-friends” attack
Thumbs down screen image on Shutterstock
Network Defender – Computer network defense engineer: “This story goes back to my Windows NT 4.0 days. It was at the twilight of my Naval career. I was the lead system administrator for the unclassified network on my final ship. We had a script that would parse through the proxy logs looking for obvious ‘bad’ site keywords. The output of the script would identify the username, machine name, and site visited.
We had two different proxy server policies that would be used, ‘in port’ and ‘underway’. When we were in port, we had a very wide pipe and everyone had access to the internet. Underway, we were very limited on bandwidth and access was limited to people needing it for ‘official business’ only. Sailors who did not have this level of access were out of luck, unless of course they could get the username\password combination from a mate who had official access.
One Monday morning, I was reviewing the output from the script and discovered that a particular machine was being used heavily for looking at ‘bad site’. The username was one of our NT Server system accounts! This account password was held only by system administrators and was never used to actually log into any server. It was allowed through the Proxy server only because it had system administrator privileges.
I did a little digging into the machine by remotely logging into it. It wasn’t running NT, but was running Windows 2000 and was attached to the domain! How was this possible? A little more digging revealed that the user had loaded Cain and Able and had cracked the SAM from the Domain controllers. He had all 5000 usernames and passwords for the network!
After turning this information over to my chain of command, the computer was confiscated and the young man got to go have a conversation with the Commanding Officer. Turns out it was his girlfriend who was surfing the ‘bad’ sites, while he was there of course.
This young man had the curiosity, tenacity, and technical ability to gain total access to our network and he was undone by his ego, showing off his ‘mad hacker skills’ to his girlfriend. Secretly, part me wanted to bring him to my shop and put him to work, since he had a greater desire to learn than 75% of the people working for me.
#3 Pull the right cable!
Confused cable guy image from Shutterstock
adcast: “So we are working on a very important and timely project that requires many of our team members to be online researching and typing away at the keyboard while online (doesn’t it seem that this is when it usually happens). Anyhow, our internet access goes offline and we start to see what might have happened. Equipment failure? Cyber attack? ISP having issues? No one is in the datacenter by the time we go in to check the equipment which was fairly quick.
By this time, the culprit had returned and confessed that he accidentally unplugged a cable on the switch that leads to the firewall device but plugged it back in. Unfortunately, the cable wasn’t labelled so he chose a random available port on the switch which was turned off. Once the right port was discovered and the cable plugged in, we were back in business.
It has made for a good laugh now that enough time has passed to heal the frustration in the moment. And we were reminded on the importance of a good label on the cable.
#4 Network mysteries
Detective image on Shutterstock
Rschroeder – Network analyst: “Maybe scarier, in a different way, was when I opened up my Novell network browser back in the late ’90’s and found multiple new servers with print services available on my network. On a subnet I didn’t use internally.
On a hunch I tried pinging their .1 address and found a reply. “So who’s on my network?” I wondered. I opened a telnet session to .1 to see if might be a router. “Ah, a recognised router prompt!” says I. “And it looks like a default router prompt for a 3Com Netbuilder Router–which I’m trained on. I wonder . . .”
Well, the default username was in place, along with the default password. So now I’m root on someone else’s router. Let’s see who the neighbors are.
Uh oh. Local government. Police department. Finance. Who’s the RIP neighbours? State Government! Not good.
Not wanting to probe any deeper, I got on the phone to that Local government’s IT department and found there was no one there who was responsible for their routers. They contract that out to a private company–the same one who was doing my company’s WAN services!
Okay. I informed them of what I’d found and it went right over their heads. I told them there was no security, default usernames and passwords on their routers, and I could see their departments and their access into the State government networks. They still had no clue this was bad.
So I called up my WAN service provider, asked them why that network was spanning into mine. Why they had no security.
(mumble-mumble . . . I’ll get back to you shortly!) and they hung up on me.
I watched and pretty soon those multiple servers were no longer showing up in my Novell world, The oddball subnet disappeared.
Later I spoke informally with one of the Network Engineers for that provider, and he admitted ‘One of the guys spanned the VLAN into one of your trunked ports. You both use RIP, you learned their routes. You have security enabled and your ACL’s prevented them from seeing you. We dropped the ball for their security, both in VLAN port spanning and credentials not being changed. Thanks for letting us know – and for not telling them!’
Maybe that’s the scariest part.”
Do you have any IT horror stories you’d like to share? Let us know in the comments.
This story has been updated since its original publication.