Oh Man, What A Week

How to even begin?

Last weekend, Mary and I spread over 15 tons of gravel (3/4 minus). No, we didn't haul it by hand, we rented a bobcat thingy to do the heavy lifting. We still put in nearly 14 hours of work (each) over the two days. So, the base layer for the path and patio is almost finished. It takes a lot of work to get everything level, and we're not quite there just yet. We need even more gravel. We also buried some hoses out to various parts in the plant beds (a poor-man's sprinkler system).

We're also getting a cob wall with a fireplace on the south side of our patio. Mary is trading medical services for the cob wall. The guy who builds the wall stopped by and gave us some ideas - and told us what we need to do to get ready. That should be a really neat addition to our side-yard.

This weekend we took it a little easier - as the pavers we want are not all available. I ordered a little too late, and they don't have enough in stock to fill our order. So, we spent time planting some plants in the bed, and getting some round river rock for the base of the cob wall.

We're happy with the progress being made. This weekend isn't going to be a marathon work-session, which is nice. We worked a little longer today (Saturday) than we intended, but it felt more leisurely. Plus, we stopped for a nice lunch and watched a few of the kinetic sculptures pedal on by the house (right about where the people to the right of the elephant are on this map).

However, this was the work week from hell. I've ranted before about how work has been .... sub-optimal. But this week things sort of came to a head. It's odd, because in some ways the week went well, but in others, it was just horrific.

You see, we're just trying to release a version of our software to the customer. Sounds simple. You have a deadline, you stop programming a few weeks before the deadline and just test, test test - fixing any bugs that crop up. Well, we've had this moving deadline for nearly a month now. We're always going to release "two days from now." Plus, we're pretty much coding non-stop. It's all good stuff to have, but it's not generally what you want to do right before releasing.

An old boss of mine says this whole debacle is because expectations were set improperly. That could be. There's also a clash of personalities between some of the managers - and the relationships have become caustically confrontational. All of this leads to a pressure-cooker environment.

So, this week is the "drop-dead" release date. We can no longer push out a release, it's either all or nothing. Of course, we'd heard this kind of thing before, but this one seemed to be a little more true. Luckily, we'd gotten most of the problems fixed, and stuff looked promising last Friday (just over a week ago).

Murphy's law, however, was waiting over the weekend, and bit us full on in the arse. We had an inkling of a really odd problem that occasionally caused a crash. Over the weekend it became a little more regular, but Tuesday night it hit us full-on, causing nearly every run to crash. The odd thing was that it only crashed when you ran from the end tool. Running it with exactly the same input (but not spawned from an end tool) resulted in no crash. This pointed to a memory problem, which happen to be some of the more difficult to solve. I stayed up late getting all the regression runs going (and failing), documenting what the problem was and pointing people toward the steps needed to solve it. I'd already stayed up late Monday night, making the drive up to Hillsboro on just a few hours of sleep, and late Tuesday night I'm struggling to stay awake long enough to send out the email telling everyone of the stench of the pile of poo we were in. It took me over half an hour to send out that email because I kept on falling asleep while typing it. I blame Tim's comfy couch as a main contributor.

So, while the results of the runs all looked good (enough), we couldn't release because we were crashing when it counted. A couple of us all converged upon the issue at the same time and got it solved on Wednesday. We got a couple of other last-minute fixes in by 8pm, and after running a quick errand, I started the builds at 9:30pm. I had to do a bunch of administrative things with the revision control system, so the regressions couldn't start until those were done. Yet another last-minute fix came in at 12:30, so I patched that in and released another build in the middle of the regressions (causing one to die, but that's ok). Two or two-thirty in the morning I leave to catch some zzzz's, knowing that things looked pretty good for a Thursday release.

I wake up Thursday morning, roll into work, and the results look good. No crashes, the new features added in the day before appeared to work just fine. We're in business. The new guy finds an embarrassing bug that somehow didn't get noticed during the regressions, so we fix it quickly (one liner change, luckily very easy and very safe to make). And the release is ready.

My trusty intern finishes all the spreadsheets comparing our tool with the old one, breaking down the overall comparison into design-by-design. All of the outliers are examined and noted they behaved as expected. One lingering crash is discovered, I find the reason and make note of it in the release notes.

Thursday afternoon rolls past, things look great (well, good enough). I talk with the boss (who's about to hop on a flight and return from Santa Clara), and he says to release if I'm comfortable. I get a little nervous, but with some confidence I send out the email. I'm done. The release is made. I get to go home and see Mary. No more stress from work. It's been three months in the making, but we should finally be on good ground again.

I should have known better.

Friday comes along, and there's some problem. I don't even know what it is. It's not with my customer, it's with a different one. Some new requirement has cropped up w/out much notice, and of course there has been no testing of that scenario. So the boss is running around frantic, I'm supposed to help another guy solve whatever this issue is. Only thing is, he's not even testing the newest release because the customer doesn't want the new release.

Things get worse from there. I get all hypochondriacal (yes, that's a word). Plus, the customer with a problem sends out an inflammatory email saying, "[my] product isn't working for customers A, B, and C, we advise against using it at all." Which gets by boss even more worked up. I get pretty annoyed at that as well, b/c my customer (B) is actually going to be very pleased with this release (they've already seen an (unauthorized - don't even get me started with that) copy and were very encouraged with what they saw.

I piss the boss off twice (well, probably three times) and stop working at 3pm (later than the 1pm I'd expected and wanted). The third way I pissed him off was disappearing at 3. There was nothing I could do at that point, so I emailed him saying as much and logged off. There was no way in hell I was working this weekend, and I was more and more sure that I wasn't going to make any more "last minute" changes because I'd already made the release.

The second way I pissed the boss off was sending an email directly to one of the customers saying, "oh, what I gave you will work, but it'd be better if you add X to it - you'll be more comfortable with that." evidentially, he hadn't told them much yet and having this snippet out of his control was maddening. He told me that "this is the kind of thing that makes them think we don't know how to make releases." Huh? I don't think so, it was a small thing I was telling them about - they'd probably be relieved I told them about it - very minor. Unlike the fiasco my boss was trying to pull by getting some more "last minute" changes into the release. THAT is exactly what makes them think we can't do releases properly. And they're right.

The first thing I did to piss the boss off was tell him to talk to the guy I was supposed to be helping insert the "last minute" fixes. My co-worker wasn't on the same page as the boss was expecting - so there really wasn't much I could do. The boss responded, "don't tell me to talk to anyone." I could tell he was well past his boiling point. I think I straightened things out with my response, but who knows, and to some extent, who cares?

I think we needed to make this release. The team is going nuts. The pace is hurried and hectic, and nobody is happy. I don't expect work to always be a happy place - there's bound to be some stress now and then, but 3 months solid of stress is way, way beyond what is normal/healthy/expected/permissible.

Next week should be interesting. In my head, I've already gone over some of the conversations that will probably happen. Of course, they never happen how I envisioned.

We released, that's my story, and I'm sticking to it.

Thanks for reading this far, I needed to vent (again).