Archive for the ‘Rants’ category

Lost supervisor password on a Lenovo laptop

April 20th, 2010

One of our little cherubs decided to put a supervisor(bios) password on a laptop that didn’t have the supervisor password already set.  It has been a few years since I’ve had to clear one of these passwords as I don’t do much in the way of workstation helpdesk tasks anymore.

In the old days, you just moved the jumper for the the CMOS reset and that was pretty much it.

On these new laptops, apparently the only way to reset the password, is to replace the motherboard.   Yes, that’s not a typo.  You have replace the whole effing motherboard!  Who came up with that design?  If the ignition core on my car breaks, do I need to get a new engine?  I don’t even know what to say, it’s crazy.  What possible reason could there be to have such a horrid design flaw?

Update:  I found this article on The Register confirming the absurd policy.

http://www.theregister.co.uk/2010/03/01/thinkpad_password_reset_hard_luck/

Resolution: WON’T FIX

February 23rd, 2010

SMTP notification configured to be sent from Backup Exec for Windows Servers, requiring authentication to the SMTP server, fails to be sent:

http://seer.entsupport.symantec.com/docs/290886.htm

Basically, it says, “we’re not fixing it”.  Why the hell do you put an option into a program, that doesn’t work and you acknowledge doesn’t work, and don’t plan on fixing it?

Sure, that phone will ring, but too bad if you want to dial with it as there are no numbers and we don’t plan on adding any numbers to it.  Why do we put up with this crap?

Cell phones are too expensive

February 8th, 2010

My wife and I have phones on Verizon wireless.  She pays roughly 80 and I pay about 65.  We don’t have data plans.  I started looking at upgrading our phones and maybe even going towards data plans.  A family plan would work fine for us since she uses more minutes than I do.  I don’t even need what I have(my $39.99 plan with over 50% extra in horse shit surcharges).  All together with a two year contract, two droids, and a data plan, we’re looking at $200.  That’s even before the aforementioned surcharges and other hidden fees appear on the bill.

I understand why they cost so much.  Basically, it’s what the people will pay.  Same goes for cell phones.  I’m surprised that modern smart phones such as the Droid, Nexus, and iPhone cost over $500 considering you can get a less than terrible laptop for the same price.  For political reasons at work, they won’t pay for a data plan.  I’m probably the only admin left that doesn’t get email on my phone.  :)

At any rate, do you have a smart phone with a data plan?  Does your job pay for it or do you foot the bill yourself?  Is it even worth having a data plan when you get past the novelty and wow factor?

Needle/Haystack

December 2nd, 2009

The worst documented piece of technology we have at our organization is the phone system, or rather, the analog piece of it.  In our older buildings, there is miles of twisted pair copper, numerous 66 and 110 blocks, and sparse documentation or labeling on anything.  Thankfully, the bulk of it is dead, the remainder of multiple phone systems of the past.

Now, 99.9% of our handsets are cisco VoIP phones.  We still have POTS lines dedicated to fire, burglar, fax, and who knows what else. The “who knows what else” is the problem I have.  There’s spotty documentation on what each POTS line is for, and due to many interconnected blocks spread out throughout every building, it’s difficult to determine where these lines go to from the demarc.  The most frustrating part is, out facilities department can’t tell me what equipment they even use that pulls dial tone.

At least with email systems and directory services you can sort of put a complete picture together of your servers, or using your routing tables and cisco network assistant (if you’re on cisco gear) you can put together a map of your network starting with no documentation.  Old twisted pair phone cabling is like the wild west.

Any other system or network admins out there have to deal with spaghetti soup legacy phone systems?

Lenovo SL500 Realtek NIC issue

September 3rd, 2009

We recently purchased 100+ Lenovo Thinkpad SL500 units.  We ran into a major issue with a large chunk of them having issues connecting to our LAN via the wired Realtek NIC.  It turns out there are a couple of fixes, none of which include a driver or BIOS update.  Either the IO card has to be completely replaced, or a capacitor has to be soldered onto the IO card.

The symptoms are that when you are on AC power, the NIC will connect at 10/mbps and half duplex.  If you force the switchport to 100 full and the computer to 100 full, the unit simply won’t connect at all.  The funny part is, if you run the laptop on battery, there are no issues.

Dealing with the tier 1 techs was a bit of a headache.  The high level techs at Lenovo have identified the issue but I guess it hasn’t worked its way down to the front line support.  Tier 1 support actually wanted me to return all of the laptops because we didn’t purchase on site support.  This was the first time I’ve ever yelled at any support person on the phone, which I generally don’t do because most of the issues I have aren’t their fault at all, but I lost it this time.  Finally after about 5 business days of constant badgering and being slowly pushed up the support tree, I finally got on the phone with someone who could make things happen.  And they did make things happen, FAST.  I was up against the wall to deploy these units and they flew a tech out that night to start working on the issue.

The issue was completely over the tier 1 support and I can’t really blame them as they probably don’t often have someone call with more than a 100 busted laptops.  I can blame them for not moving up the tree quick enough, but in the end, the issue was resolved.  I’m impressed that Lenovo didn’t leave me in the rain and are sticking by their product and their mistake.  Too bad software companies don’t follow suit when they release shit software and charge hundreds of dollars to even talk to an engineer.

Red lights of impending doom

June 22nd, 2009

I walked in this morning and got logged in.  Shortly after that, I was approached by a coworker stating that the server room was really loud.  I wandered down and heard the server fans screaming away before I even got to the door.  As soon as I opened the door I was blasted by a head wave as hot as some of our mid summer heat waves here in eastern PA.  At first, I felt like I was walking into some crazy bizarro server room.  Every server was flashing amber or red and there was a slight odor of burning electronics.

I quickly realized the 1 year old AC unit had crapped out. The server room has two doors so I opened both doors and grabbed a few box fans to start moving air through the room. I then started to shut down most of the servers and a large chunk of the network gear. Somewhere during that, I called our facilities department and alerted them of the situation. They began to work on the AC unit and determined that a fuse blew, which has been replaced and the unit is back on. The server room is still cooling down now and I have slowly been bringing servers and other equipment back online.

There are a few issues here. Ultimately, things break and we need to have some sort of control of the environment of that room or at the very least, a way to monitor it. First of all, I should have an independent temperature sensor that will alert me when it gets too hot in there. If someone hadn’t alerted me to the problem when they did, there could have been a shit load of issues if equipment started to fail.

The other issue is that I’ve always been toying around with a network monitor system, but I’ve never been able to settle on anything. I’ve mucked around with Zenoss and nagios but I’ve never stuck with anything. I need to just find a proper system and knuckle down and get it configured.

There are lessons to be learned from this morning!

iScsi SAN or NAS with VMWare

May 27th, 2009

A while back I posted something about iScsi SANS.  What I’d really like to see, once all is said and done, is to be able to virtualize almost every server we have.  I’ve been eyeing up the Lefthand redundant storage solution but the price is always an issue.  Up until recently, the likelyhood of moving forward with virtualizing our servers and storage has been out of the question due to funding.  Now, I’ve seen a small crack of light from the door accidently being opened up a crack so I’m just going to pretend it’s going to open and I’ll be showered with budget numbers and blank purchase orders.

I just noticed a post by a netapp employee, Dave.  I haven’t thought of running VMWare over NFS to a NAS.  To me, without looking at pricing*.  Any thoughts on this?

*Sales people, when someone from a K-12 Public School District calls for pricing, it’s not because they’re ready to buy.  We’re calling because if we can’t afford it, we’re not buying it.  I don’t even bother getting too deep into a possible project without knowing the pricing because that’s the bottom line regardless of features.  To the bosses, it’s as simple as why and how much?  When I see “call for pricing”, I cringe.  When someone tells me I have to sign an NDA to get pricing, goodbye.

Apparently, the best way to fix a problem…

May 13th, 2009

…is to walk away from it for about a month. In this case anyway.

We had a lot of legacy cruft in our network that has been and continues to be slowly phased out.  We have recently replaced our content filter running Bordermanager with a Barracuda 610 web filter appliance…sort of.  There is a legacy web based app which we have to run through the summer.  When I cut everyone over to the Barracuda away from Bordermanager, it broke the web application.  Right now, students go directly through the Barracuda, everyone else goes through the old proxy box which in turn goes through the Barracuda.  That wasn’t really a large issue, but the proxy server has been locking up cold lately.  The only thing that really changes on that box is the web cache processed by the proxy nlm.  Maybe it was choking on something there…or I’m just grasping at straws.  Either way, I don’t think it matters much, since ultimately that box needs to go away.

I finally got to spend a couple of minutes on the problem today.  NSLookups on the hostname worked fine.  But I couldn’t ping it from my test machine.  I could ping it from my own machine without any issues.  I figured out what was going on.  The web app server is on the old legacy 10.0.0.0/8 network where my workstation and the proxy server all reside.  So, routing is no issue there.  The default gateway of the proxy and my machine is our core 4507.  Unfortunatly, due to more legacy issues, the default gateway of the web app server is an old pfsense firewall connected hooked to our old ISP connection so it can’t have a proper gateway to the inside of the network.  That goofy setup is only around because of DNS issues and the fact that Oracle application servers, at least our version, are incredibly stupid and need a complete reload if you want to change the hostname.  There’s more to that story, but suffice it to say, we have to keep this setup until the summer.

To fix the whole mess, or rather duct tape it together, I just configured static routes into pfsense for the rest of the network.  So, maybe walking away for an hour would have worked, I just chose a month.  Also, this is a testament to keeping things simple.

“Breaking” my laptop

March 13th, 2009

I made the mistake of creating a DHCP exception based on the MAC address of my wired interface on my laptop.  I promptly forgot I did it.  It wouldn’t be an issue in a small environment, but we have roughly a dozen data access vlans which could be recieving DHCP addresses.  I kept running into the problem where at work, I couldn’t get an address from our DHCP server.  The complicate things further, but ultimately made me realize what the problem was, both my Windows and Linux installs bombed out pulling an address.  I knew the issue wasn’t the actual NIC as it worked fine at home and on my little pix 501 test box.  Yesterday I realized that I had put the exception into one of the DHCP scopes.

When my laptop tried to pull an address, it would see the DHCP exception and try to assign it a specific address.  That address was in a VLAN that wasn’t allowed in most of buildings/areas.  It’s always fun to sabotage yourself!

Verizon, ugh

December 3rd, 2008

In what seems to be a common theme with voice and data providers, I find it’s impossible to find anyone at these companies who know what any other part of the company is doing and who is responsible for what.  Verizon “owns” the DNS records for our domain at the moment and the person who handled all of that for us is not longer working at our organization.  I’ve been trying to get in touch with our sales “rep”, who comically won’t answer her phone and her voicemail box is full.  Prior to that, I was given the run around by someone at Verizon support who claimed they needed to snail mail me a form which would take 2 weeks to get to me.  It’s been over a month and I haven’t recieved anything. Does it have to be this hard to get anything done?