Jim's Random Notes

I'm writing a Web crawler.  Yeah, I know.  It's already been done.  It seems like everybody's done some Web crawling.  But there's a huge difference between dabbling at it and writing a scalable, high-performance Web crawler that can pull down hundreds or thousands of documents per second.  Granted, processing more than a few dozen documents per second will require a heck of a lot more bandwidth than I currently have, and if I want to get to the thousands I'll need multiple machines working together.

But first things first.  Before you can scale up, you need something that works no matter how slow it is.  This turns out to be more difficult than it seems at first glance.  The idea is very simple:  read a Web page, scan it for links, put those links in a queue, grab a link from the queue, lather, rinse, repeat.  Ten billion times.

Ten billion is a big number.  If you process 20 documents per second, it'll only take you 500 million seconds to finish the job.  That's almost 16 years.  Obviously, you need to process things much faster than that.  It's a bandwidth issue and a machine optimization issue.  Let's look at both.

The easy part is bandwidth.  It's hard to get an authoritative figure, but a good round number for average Web page size (HTML only) is about 25 kilobytes.  The best throughput I've seen on my cable modem is about 5 megabits (approx 500 kilobytes) per second.  So figure a sustained upper limit of 20 pages per second.  That's 20 HTML pages downloaded and links extracted.  Doesn't seem so tough, right?  I mean, a 2 GHz computer with a 100 megabit network card can easily keep up with that.

It's harder than it sounds.  Not because the processor can't keep up, but because it's incredibly difficult to saturate the bandwidth.  There's a whole lot of latency involved in reading Web pages.  Most of the time, the computer is just sitting there waiting for servers to return data.  The solution is to keep multiple connections going so that while some are waiting, others are pulling down data.  That turns out to be a non-trivial task.  First, because managing dozens or hundreds of concurrent connections is hard, and second because domain name resolution takes an absurdly long time.  And it's often implemented very inefficiently.

There are many other issues you have to think about if you want to crawl the Web.  You'd think that keeping track of the URLs you've visited and those that you haven't visited yet would be easy.  And it would be if the numbers were manageable.  10 billion documents means 10 billion URLs.  Just the URLs themselves, stored naively, would take close to a terabyte.  So you have to come up with a memory-efficient way to store them and a time-efficient way to search them.

There are other issues as well.  For example, did you know that http://www.mischel.com and http://mischel.com go to exactly the same place?  That's a trivial example, but you'll find out that some Web pages have many different URLs.  If you don't have some way of resolving those and other traps, URLs with odd query strings, unique session IDs, and any number of other oddities will quickly have you chasing your own tail.

Today I finally got my first multiple-connection Web crawler up and limping.  It took a whole lot longer than I thought it would because the asynchronous HTTP library I downloaded was buggy and it took me forever to figure out how to make Python break out of an infinite loop.  For all that's good about Python, the development and debugging environment is distressingly primitive.

All of which is to say that I've latched onto a project that is very interesting and challenging, and doesn't leave me time for much else.  When I'm not banging on code, I'm reading research papers about crawling, storing, indexing, and searching large amounts of data, or thinking about better ways to implement parts of my crawler.  It's exhausting, but pleasantly so.

I've changed hosting providers from SectorLink to lunarpages.  These things don't happen overnight, so it might be a few days before DNS propagation has everybody seeing the new site.

If you're seeing this page it means that everything's working for you.  This page is on the new site.

It's possible that my email is not working yet.  If you get a bounce back from sending mail to me or to Debra, please try again in a couple of days.

A new project has me diving into Python.  Python is an interesting and somewhat quirky language:  quirky in that it does some things much differently than how I've seen them done in other programming languages.  With just a couple of days' experience, I can say that I like the model more than I like PHP, mostly because Python shows more forethought--design--whereas PHP looks like it started as a clever hack that evolved.  I'll be the first to admit that this is a totally subjective opinion that's based on very little familiarity with either language.

I ran into an interesting problem today when trying to get the urlgrabber package working.  I downloaded and installed the Python module, ran the unit tests, and was told that everything worked.  But when I tried to run the example program, called urlgrabber, I kept getting an error message trying to import urlgrabber.grabber.  This was very odd because the sample programs had no problem importing that module.

30 minutes searching the Web didn't reveal a solution.  It finally dawned on me that the problem might be that there might be a name resolution conflict.  The program called urlgrabber was trying to reference a module called urlgrabber.  This probably isn't a problem under Linux, but under Windows it appears that Python was looking in the current working directory for the urlgrabber module.  What it found instead was the urlgrabber program itself.  It then searched for grabber in that directory and didn't find it.

I confirmed my suspicion by renaming urlgrabber.py to xxx.py.  The program works fine now.

This all makes sense.  Windows always searches the current working directory first when looking for programs, DLLs, and other things.  Only after that does it start searching the directories identified in the PATH and in other places.  Linux, on the other hand, does things differently.  In particular, you can configure Linux so that it specifically does not search the current working directory.  This is very common, in fact, when running as root.

Problem solved.  For now.  On to the next one.

The enabled property of the flash.display.SimpleButton class, according to the Flex Language Reference, is supposed to enable and disable a button.  The documentation reads:

A Boolean value that specifies whether a button is enabled. When a button is disabled (the enabled property is set to false), the button is visible but cannot be clicked. The default value is true. This property is useful if you want to disable part of your navigation; for example, you might want to disable a button in the currently displayed page so that it can't be clicked and the page cannot be reloaded.

Unfortunately, it doesn't seem to work.

If I set enabled to false, the button no longer responds to the mouse moving over it.  That is, the button states don't change when the user moves over or clicks on the button.  But a click on the button does result in a CLICK event being sent to the event handler.  So although enabled prevents the display state from changing, it does not prevent the button from raising events.  In other words, enabled is useless.

However, you can disable a button by setting the hitTestState to null.  The Flash runtime tests the mouse position against the hitTestState object to determine if the mouse is over the button.  hitTestState is usually set to the same value as upState, but if hitTestState is null then the button has no presence as far as the hit testing code is concerned, making it impossible to roll over or click on the button.

So I wrote the following little function to enable or disable a SimpleButton object:

private function enableButton(button:SimpleButton, bEnable : Boolean) : void
{
    if (bEnable)
	button.hitTestState = button.upState;
    else
	button.hitTestState = null;
}

It seems like the enabled property should work as advertised. I wonder if I have an older version of the Flash 9 runtime, and that bug is fixed in an update. I guess I'll have to go hunting.

The HyperBike is the ultimate human-powered land vehicle.  The thing is a work of art and an apparently effective transportation device.  The inventor swears that you could get 50 MPH out of the HyperBike, although you couldn't convince me of that by how he operates the thing in a parking lot (in this video).  It looks exceedingly awkward.  The cranks for the foot pedals look too short, and the standing position doesn't look like the most effective way to operate the hand cranks.

But for all that, the HyperBike is really cool.  And the model posing with it isn't too bad, either.

Somebody in a car wouldn't able to say, "I didn't see it!", about one of these things coming down the road.  And if 50 MPH is easily achievable, I should be able to keep it at 30 or 40 without too much trouble.  A sustained 30 MPH over a flat course would be very nice.  I can't do that on my road bike for more than a few minutes.

Now where do I strap the grocery container so I can take this thing shopping?

I've been doing quite a bit of ActionScript coding recently, trying to complete a game I started some months ago.  More than anything, I need to complete a project.  I've been directionless for far too long, which has contributed to my recent mental state.  Focusing on this game--on completing the game--has helped me start to regain a little balance.  Although I'd like something to become of the game, the most important part right now is completing it.

The ActionScript language itself, as I've mentioned before, is quite reasonable; in most cases, a joy to work with.  It has rough edges, as do all programming languages, but overall I very much enjoy working with it.  The Flash environment, too, is nice, but I'm continually bumping my head against the "this isn't Windows" problem.  The Flash model is fundamentally different than Microsoft Windows and, whereas there are many similarities, the differences sometimes confound me.

The Flash event model, for example, is much closer to X Window than to Microsoft Windows.  Although I think I prefer the Flash model, my unfamiliarity with it has been the source of many head-scratching moments.  It does slow down the development process, but I also have very much enjoyed learning new things.

What's giving me trouble right now is that the Flash APIs don't appear to include the concept of a modal dialog box.  The Flex framework does, but at a cost.  Even the smallest Flex program is over 100 kilobytes in size.  Any meaningful program that uses Flex will quickly suck in more than 200 kilobytes of framework code.  That's just too much.  My game, which does not use Flex, is only about 220 K in size.  That includes all of the sounds and artwork.  Using Flex would probably double the size of my program.

Some may think it's silly, what with the wide availability of broadband Internet access, to be worried about adding 200 kilobytes to the download.  Afterall, bandwidth tests on my connection here at home report a download speed of about 600 kilobytes (yes, bytes, not bits) per second.  So 200 K takes roughly 0.33 seconds to download.  And 200 K means almost nothing in terms of storage these days, when computers regularly come with 80 gigabyte drives.

There is an issue on the server end, though.  Bandwidth charges start to add up when thousands of people download a program that's twice the size that it has to be.  It's hard to say if that's really an issue.  Some providers give two gigabytes of bandwidth per month with their basic hosting packages.  It would take 5,000 downloads to exceed the limit.

Some of my reluctance is undoubtedly aesthetic--the old assembly language programmer in me who hates to see all those wasted code bytes.  Whatever the case, I've committed to making this game without using Flex.  After that's done, I'll re-evaluate my options before tackling another Flash project.

One day I told my friend Dean that I regretted something I had done. He looked up from what he was doing and said, "You should only regret those things in life that you don't do."

That stopped me in my tracks. I pondered it for a minute while I stared at Dean, and finally said, "You know? You're right."

That little lesson has stuck with me ever since, and 25 years later I can remember almost everything about the conversation: where we were, what we were doing, and the way this simple statement hit me. It was like a revelation, and something I've tried to live by. Every time I talk to older people, they speak of regretting not doing things. Very seldom does somebody say, "I regret doing this." But all too often they say, "I regret not doing that."

The more interesting thing is why people end up in a position where they look back and say, "I wish had..." The most common reason is fear: fear of taking a risk and failing; fear of being seen as odd; even fear of success, oddly enough, because we don't know what we'd do if our plans succeed. Those and most other common fears can be summed up in one: fear of expanding one's comfort zone.

People build comfort zones out of necessity. We are, after all, creatures of habit. We create routines, schedules, quirky ways of doing things, frames of reference through which we filter information, and little rituals that help us operate on a day to day basis. Without those things--our comfort zones--we would be overwhelmed by everyday activities.

Even within our comfort zones, there are things that are distinctly uncomfortable, and that we avoid if at all possible. The more we avoid those uncomfortable things, the more uncomfortable they become. Something that you may have found mildly distasteful early in your life can become very frightening later in life if you spend years avoiding it. Your comfort zone shrinks, and continues to shrink as you avoid more things. The shrunken comfort zone is why so many people remain where they are, even when given the opportunity to move on to something better. Their fear of the unknown overrides their desire for a better situation.

The only way to prevent your comfort zone from shrinking is to continually work at expanding it. That's why I'm always looking for new and different things to try: learning to brew beer, bicycle touring, Toastmasters, replacing the engine in my truck, ham radio, and any of dozens more activities I've dabbled in over the years. By facing the fear and uncertainty that comes with learning new things, I expand my comfort zone and make it easier to learn the next new thing.

Unfortunately, that's not the whole answer. You see, we don't have just one comfort zone. We have many. I, for example, am very comfortable trying anything new physically, and I love learning new things. I'm game to try my hand at anything mechanical or electronic. I'm not especially good, but I can muddle through and learn a bit in the process. Give me a physical or mental challenge and I'm quite happy to attack it.

But I have my blind spots, too. I have a distaste for paperwork and bureaucratic routine that borders on the pathological. I'm perfectly willing to pay money--a lot of money--to have somebody else deal with those things. And my fear of emotional conflict quite likely is pathological. I have avoided conflict for decades. Today, even the thought of emotional conflict stirs up the "fight or flight" response in me like you wouldn't believe. The lengths to which I'll go in order to avoid conflict are astounding. When it comes to conflict, my comfort zone is very, very small. I've been working on that since I noticed a few years ago just how bad it had become, but I have a long, long way to go.

The sad thing is that I've known all of this about comfort zones for at least half my life. I've even noticed and commented to friends over the past few years that I've found it hard to face certain challenges--things that I would have had no problem with even five years ago. And yet I've let my comfort zones shrink to the point that I spent six months avoiding some opportunities that were pounding on my door. I don't know why, exactly. Fear of failure? I had nothing to lose. Fear of success? Perhaps. Fear of the unknown--of giving up my vaguely dissatisfying life and risking everything for the chance to be happy beyond all expectation? As much as I hate to admit it, that just might be the reason.

Whatever the cause, I'm going to regret not making the most of the opportunities that I've had in the last six months. But I won't dwell on the regret or on the lost opportunities. For all the things I've done wrong recently, I've managed somehow to come out feeling better about myself and my prospects for the future than I would have thought possible.  I will undoubtedly look back and wish for what might have been, but my focus is now forward, on what I can make happen.

I mentioned last week that I'm giving serious consideration to moving my Web page to a different hosting service.  I've done a little research and have found a dizzying number of providers, all offering similar services for similar prices.  Some give more bandwidth or disk space than others, but their limits are so far beyond anything I envision needing right now.  The dizzying part is trying to figure out which of the provided software tools are best.  Everybody has different Web mail programs, blogging tools, site designers, etc.  It seems like most provide access to WordPress for blogging, and my experience with WordPress shows that one could use it for an entire (simple) Web site.  Beyond that, though, I'm lost.

My primary concern is with the Web mail software that the host provides.  It didn't take me long at all to get used to checking my mail online, and I rather like the idea of keeping everything there so that I can check mail from any computer connected to the Web.  If I store all my messages and keep my address book online, I can do stay in touch anywhere.  But the email client has to be reasonable.  Certainly it has to be better than the new version of SmarterMail, which has so many little annoyances that it's almost unusable.

Yahoo's new Web mail client, on the other hand, is just amazing.  Of the few Web mail clients I've used, it comes the closest to acting like an installed mail client.  The interface responds quickly, the commands are mostly intuitive, the calendar and address book are well integrated, and--perhaps most importantly--the program doesn't surprise me, natter at me, or get in my way.  It's a simple but powerful mail client, and has gone a long way in changing my opinion of Web applications in general.  Unfortunately, the only way to get Yahoo mail would be to have Yahoo host my Web site.  That's not completely out of the question, but there are other, better, hosting options.

Lunarpages, for example, offers Horde's IMP, and something called SquirrelMail.  Both look decent (I think SquirrelMail looks better), but that opinion is based solely on reading their descriptions and viewing screen shots.  Other providers supply different mail clients.  Which one is best?  Can I test them out somewhere?  I've been surprisingly unable to find reviews of Web mail clients.

Another option, I suppose, would be to use a Yahoo mail account and have all my mail forwarded there.  That's certainly an option that I'd consider, although it wouldn't be my first choice.

Any ideas?  I'd like to hear about your experiences with other hosting providers, opinions on different Web mail clients, or pros and cons of using Web mail rather than a traditional client application.

Like many (most?) people who know little about the stock market, my investing is limited to mutual funds and a "buy and hold" strategy that involves dividend-paying stocks in large companies.  Also like many people, I've made some forays into trying to make money fast by buying a fast-rising stock or gambling on an exciting startup.  As you might expect, I had some wins and took some lumps.  I'm a fast learner, though, and decided early on that I shouldn't be throwing my money at something I didn't understand.  Trying to make money trading stocks when you don't understand what's going on is just gambling.  Stupid.

I've done okay with my investments over the years.  Not setting the world on fire by any means, but I have more now than when I started and it doesn't take me a whole lot of time to review my holdings periodically and readjust when appropriate.  I've also nurtured a healthy distrust--one might say unreasonable fear--of "trading," to the point of being convinced that it is nothing more than gambling.  I've recently had occasion to rethink that opinion, so I thought I'd take a closer look at it.

Understand up front:  nothing in the discussion below is intended as encouragement.  You will never see me recommend that somebody take up short term trading in the stock market.  At this point in my research I'm not even sure that I'll attempt it.  I'm just interested in learning more about how it works.  If it works.

A "buy and hold" investor purchases shares of a company's stock based on the company's long-term growth prospects.  They expect the company's value to increase and the stock price to increase along with it.  In addition, the stockholders participate in the company's profits through quarterly dividends.  This kind of investing is seen by many as the "right" way to do it.  If you do your research, buy stock in a quality company at a good value and hold it for a long period (years), you can do very well.  Warren Buffet has proven that spectacularly, and others have to a lesser degree.

Traders are completely different.  They attempt to make money by taking advantage of volatility in the market.  The money they make comes not from the companies whose stock they buy, but rather from other traders who lose money.  Or from ordinary people (like me) who make stupid gambles and lose.  The basic tool of the trader is called technical analysis, which has received a bad rap over the years.  That bad rap is, I think, mostly undeserved.

Technical analysis is based on the observation that in the short term stock prices move in indetifiable patterns that are based mostly on the emotions of greed and fear.  The long-term prospects of a company do come into play, but act mostly as motivators for one of those two emotions.  For example, a rumor might surface that Jim's Unique Kryptonite (JUNK) will announce a new product.  Greedy investors jump into the stock, hoping that the new product will increase the company's revenues.  More demand drives up the price and more buyers jump onto the bandwagon.  At some point potential buyers see the price as too high or traders decide to take profits, and the trend reverses:  people begin selling the stock, the price begins to drop, and more holders begin to sell because they fear the price will drop below what they bought it at.

That's oversimplified, of course, but it's essentially how stock prices fluctuate over the short term:  greed drives prices up and fear pushes prices down.  In extreme cases you'll see things like the late 90s tech stock boom, where prices went rocketing up beyond all reason, only to come crashing down even faster when people finally realized that they were holding $2 worth of stuff that they paid $10 for.  But even in a relatively stagnant market, prices will often fluctuate ten percent or more over a relatively short period.  Short-term traders attempt to take advantage of and make money from those fluctuations.

Make no mistake, short-term trading is a competition.  Not every dollar made by short-term traders is offset by somebody else's loss (share prices do tend to rise in the long term), but one could safely say that most of the profits are gained at somebody else's expense.  If you understand that, then you're one step ahead of many others.  Those who continually come out on top in any competition do it through hard work, not luck.

Technical analysis is how traders refer to the wide range of analysis tools that they use to predict the movements of stock prices.  Analysts look at things like volume, moving averages, candlestick charts, oscillators, and untold others to spot trends and, more importantly, impending trend reversals.  Their intent is to catch a stock just as its price begins to move up, and then to sell it just as the price begins to move down.  Or, in reverse, sell a stock short just as the price begins to decline, and then buy it back (cover the short) just before the price begins to rise again.  The time period involved is usually three days to six weeks.  Day traders have a much shorter time scale (minutes to hours) and use some different indicators, but they work in essentially the same way.

My question, when I began researching this, is whether or not there is actually some validity to all those numbers, charts, averages, trend lines, and so on.  What I've discovered so far is that there does appear to be something there, although like anything it's not as simple as it first seems.  Stock prices do move in recognizable patterns over the short term, and it's possible to predict those patterns with some degree of accuracy.  Sometimes.  No trader is always right.  In fact, some successful traders are wrong more often than they're right, but with tight money management they're able to minimize their losses, maximize their gains, and still come out ahead.

Based on everything I've read so far, I'd say that it is possible to make money by short-term trading.  It's not easy.  It requires patience and study as well as discipline.  A lot of discipline.  And that's where I think most of the amateurs go wrong.  They don't have the patience to really study and understand what they're doing, and they lack the discipline to develop and implement a coherent and workable plan.  Without those things, a trader is only a gambler, and he's playing a game that he can't win against people who are very motivated and quite serious.  He can only get lucky once in a while.

The "bad rap" that technical analysis has received appears to be the result of people who dabbled in it without understanding what they were doing.  Typically unwilling to accept the blame for losing their money, these people instead try to offload the responsibility.  "I tried that technical analysis crap.  I lost all my money."  Funny thing is that they credit their own genius for their gains.

As it stands, there's no way I would attempt a foray into short-term trading with the little knowledge that I currently possess.  There are way too many things that I don't understand.  I'll continue to research it because I find the subject quite interesting, but right now I can't say that I'll ever study it in depth and follow closely enough to feel comfortable with attempting it.

Lots of things percolating in the brain these days.  I don't know if any of them will stick, but it feels good to be thinking again.

• Like most people, it seems, I experienced a huge spike in my spam count sometime in November or early December.  I don't know if somebody had learned to defeat front-line filters or if a surge in spam just overwhelmed things.  Whatever the case, Thunderbird's spam filter wasn't any help.  Sunday my hosting provider installed a new version of SmarterMail and apparently some new front-line filters that reduced spam to a trickle.  But yesterday they rolled back the changes and my spam count is back to almost 200 per day.
• I used my Web mail client while I was on vacation last week because my Thunderbird client couldn't send mail through the server from my hotel connection.  I don't know why it could receive mail and not send, as the password is the same for both.  In any event, I've come to appreciate the convenience of keeping my mail on the server and using a Web client.  If only my provider (SectorLink) had something better than SmarterMail.
• I'm giving serious consideration to changing hosting providers.  SectorLink is okay, although they've lost my mail at least twice, and several times there have been interruptions to my service.  Other hosting companies give more and apparently better service for a lower price, and include things like one-click installation of blogging tools, site designers, etc.  I can get all that from LunarPages, HostGator, or one of dozens of other sites for less than half of what I'm paying now.
• Some of you know that years ago I tried one of the first PDA phones, and found that I didn't much like it.  The screen was too small, data input was difficult at best, and the overall features were pretty weak.  I found that I didn't really need to get my email that way, nor did I want to read Web pages on that little device.  I have two friends, however, who say they're incredibly productive with the newer units.  One is using a BlackBerry Pearl.  It might be worth looking into again.
• I've seen several of these reports lately, of people blindly obeying the satnav or GPS navigation systems.  I can't even imagine how totally clueless somebody has to be in order to drive off the road based on the word of a computer system.  Don't they even look before turning the wheel?  How could these people even have received their driving licenses?  Don't they teach judgement anymore?  The mind fairly boggles at the idiocy of it all.

After a week of temperatures in the mid 60s to low 70s, I drove home yesterday into ever-deteriorating weather.  I was driving in rain for most of the second half of my drive--from south of San Antonio all the way home.  This morning there was ice on the truck and the door was hard to open due to icing.  As of now (about 12:30 Monday afternoon), there's a thin coating of ice on the holly bushes and it continues to rain off and on.  The roads will likely be iced over before nightfall.  The weather forecast is talking possible snow tomorrow night.

In case you haven't noticed, serious cold is relatively uncommon here in central Texas.  It freezes a couple of times per year, and every other year or so we get either snow or freezing rain that shuts down the city for a day or two.  But next week will be in the 60s again and we will have forgotten what little bit of winter we had.

Charlie is a big baby when it comes to cold.  He doesn't like rain at any time, although he will go for a walk in the rain.  I guess some things are worth getting wet for.  In any event, we have a really tough time getting him to go outside in the cold.  He'll ask to go out and then turn around and go back to his bed if he feels cold air when we open the door.  When he finally can't hold it anymore and he must go outside, he steps right outside the door, does his business, and comes running back in like something's chasing him.  For such a big and supposedly mean dog he really is a wuss when it comes to cold and rain.

This morning he did something I've never seen him do before.  He curled up in Tasha's bed.  Understand, Tasha is a six pound poodle.  Sure, the bed is a bit larger than she is, but I never expected Charlie to get his 80 pounds of wimpiness into it.  But there he is, curled up in a much smaller space than I thought was possible for a dog his size.

I set out about 11:30 this morning on my way to Port Aransas, TX, for my week-long "vacation."  The plan is to spend a few days relaxing and walking on the beach before buckling down and doing some serious soul searching to figure out what I want to do next.  I stopped by the office to pick up a few things and then headed out.  I was only about 15 miles down the road when I realized that I had forgotten the power cord for my laptop.  So back to the office to pick it up.  Not a great way to start a vacation, but that's okay.  I'm not in any big hurry.

About an hour later, just a few miles north of San Marcos, I looked back and saw a big plume of smoke following me.  "Can't be my truck," I thought, but then re-thought that when I changed lanes and the smoke stayed with me.  Not fun.  You can get almost anything on Sunday these days, but auto repair is pretty hard to come by.  I can't say for sure, but it looks like I have a hole in one of the transmission cooler lines, which results in transmission fluid pouring out.  It's probably something I could fix myself given a few tools and a place to work, but I'll stay here overnight and have my truck towed to the shop in the morning.

It's not how I wanted to start my trip, but getting upset about it wouldn't do any good.  The room is reasonably comfortable and a lot quieter than I thought it would be, given that it's right on I-35.  I took a walk, read a bit, and did a little writing.  I don't have my feet in the sand at the beach, but I'm still relaxing.  Just one more detail to take care of in the morning and then my vacation starts in earnest.

Perhaps it's true that the adventure lies in the travel, not in the destination.

2006 was a difficult year for me.  It started out well.  I got to visit Japan again in February, finished the truck's engine transplant in May, and finished the graphics editor project in June.  That project combined with some other work and careful budgeting allowed me to bank six months worth of salary in the business account, giving me some time to find and begin working on a new project.  And then my brain fell apart.  I couldn't get excited about anything.

More correctly, I could get excited about things but I couldn't maintain my excitement about anything concerning work.  I got started working with Action Script 3 and the new Flash 9 over the summer, but began struggling with motivation in late August.  I simply haven't been able to stay focused on any project that requires more than a few days--perhaps two weeks on the outside--of effort.  The only thing that got me really excited over the last six months was my birthday bike ride.  Other than that, the thought of embarking on any project that takes serious thought is painful.

If you're wondering why my posting here has slacked off so much over the past few months, that's it.  My lack of motivation has affected all aspects of my life, including my curiousity about things and my desire to write about what I've learned.

It's doubly distressing because I used to enjoy programming and writing so much.  It was how I defined myself.  To find myself actively avoiding the two activities that have given me the most enjoyment over the years bothers me more than the idea that I've basically wasted six months of my life puttering away at things while I refused to acknowledge the problem.  Even more depressing is that I thought I went through this already, five years ago.  I even wrote about it back then.

Such things can't help but affect my personal life, and when they do it's time to face the problem and find a solution.  Or at least try to determine what the problem is and begin working towards a solution.  A very strong voice in my head tells me to suck it up, quit complaining, and get back to work.  Many people do that because it's what society expects.  They'll give up their happiness in order to avoid looking weak and having to examine their own desires or face their fears.  Perhaps that's what I'll end up doing.  But not just yet.

I'm taking a little vacation alone next week, getting away to somewhere quiet and away from daily responsibilities so that I can clear my head and spend some time trying to figure out what I want to do for the second half of my life.  I can't honestly say that I'll have all the answers when I return, but I expect to have some idea of what to do next.  Perhaps the next step is therapy.  All I know right now is that I can't continue to spend all day sitting in front of the computer waiting for my motivation to return.

In his article, The Open Source Desktop Myth, author David Chisnall points out correctly that Microsoft has already won the war.  That, although Apple's OS X is superior and you can build a Free Software desktop OS that's "better" than Windows, it would be very difficult to lessen Microsoft's hold on the desktop.  He then points out, also correctly, that it doesn't really matter.

His reasoning is simple:  computing is moving away from the desktop.  Apple sells more notebook PCs than it does desktops.  Other manufacturers also are seeing notebooks outsell or coming close to outselling desktops.  It's no surprise, really.  Notebook PCs are incredibly powerful these days--plenty powerful enough to do what most users want them to do.  They're more expensive than desktops, but when the price point is under $1,000, the convenience of having a notebook far outweighs the price premium over a desktop system.

But the notebook isn't significantly different from the desktop.  They have the same operating systems and the same user interface devices.  For most users (myself most definitely included), the notebook is just a portable desktop machine.  I don't use it any differently than I used my desktop machine two years ago except that I can carry it wherever I go.

Chisnall then goes on to point out that other devices are far more ubiquitous than desktop or notebook PCs.  You can get mobile telephone/computing devices today that are much more powerful than the desktop PCs of 10 years ago.  The user interface devices are quite different, of course, but with keyboard and monitor add-ons, it's possible now to do on a mobile device (that doubles as a telephone) all of what "normal" computer users do with a computer these days.  Surf the Web, check email, play a rousing game of Free Cell, manage a calendar and address book.  With Web-based email, calendar, and address book, there's no need for local storage.

What many people don't realize--even those who say, "the Web is the future"--is that the Web is the future!  All the technology is in place for the desktop PC to become almost completely irrelevant except as an appliance to get to the Web.  With systems like Ajax and (conspicuously missed by Chisnall) Flash, the browser-based desktop is possible.  Amazon's S3 and similar offerings from others provide secure and reliable storage for Web-based applications.  Ajax and Flash provide browser-based applications that, although slower than native applications today, they're still as fast or faster--and quite a bit more feature-rich--than similar applications from five years ago.  It's entirely possible to build a Microsoft Word or Excel equivalent in Ajax or Flash and have the data files stored on S3.

It's not much of a stretch at all to envision an integrated development environment, a collaboration system, or even an entire Web-based desktop that's designed to run in the browser, storing files somewhere "out there."  At that point (and I think that point will be very soon), the device becomes irrelevant.  I should be able to go up to any computer or mobile device that has a functioning Web browser and access my desktop.  All of my files are available to me instantly, from anywhere in the world on any computing device.  I don't have to carry a notebook computer or a memory card or anything else that saves state.  Any dumb browser device will do.

There are arguments against that idea, perhaps the biggest being bandwidth limitations and the perceived vulnerability of data stored on the Web.  Bandwidth is a problem in some cases, although for most it's not at all.  Web-based email actually uses less bandwidth than does a POP client, because you don't have to download messages that you don't read.  It requires bandwidth to pull down a document that you want to work on, but most of those documents are so small as to be irrelevant.  Even music files (at about one megabyte per recorded minute) come down at reasonable speed, so online editing of audio is well within the realm of possibility today.  Video editing still requires huge files--big bandwidth if stored online.  Few other applications have similar requirements.

I suspect that your data is actually safer on S3 than it is on the normal user's desktop that has an always-on connection.  Administrators of online storage providers are much more aware of security issues than the average user.  Whereas Joe Public's computers are continually infected with worms, viruses, Trojans, and all other manner of malware--most of which the user doesn't know about--commercially-operated storage sites are hardened against intrusion and are continually tested and updated to prevent security breaches.

I've heard people argue that users won't be willing to store their precious data online, but I dispute that.  Most of us have electronic access to our bank accounts or investment accounts (ETrade, etc.), credit card statements, PayPal, Amazon, eBay, and hundreds of other sites that store our precious personal and financial data.  Every once in a while you hear about security breaches, but nobody I know personally has ever been affected by one.  The public already trusts their data to the Web.  Implicitly.  The only difference in what I'm talking about is that users will all of a sudden be explicitly trusting their data to be stored "somewhere out there."  Some will grumble, but everybody will do it.

If the above happens--and I don't see any reason why it won't--Microsoft is in a tight spot.  Microsoft is married to the traditional way of doing business.  That is, packing functionality into the operating system and other installed software:  selling software as a product, one computer at a time.  Their business model cannot survive a move to the Web, with software provided as a service.  That is, the software is essentially free, and users pay for storage or back-end functionality.  Microsoft has some of the best developers in the business, and great thinkers who have seen this coming.  They've even made some half-hearted attempts at creating such Web-based applications, but doing so hurts their core business of installed software and upgrades.  If Microsoft throws its efforts behind a "software as a service" model, they will kill their cash cows.

The face of computing is going to change.  Not as quickly as I'd like to see it change, because there are plenty of Microsoft-like companies out there insist on creating buggy whips, hoping that if they don't embrace the new way of doing things, it won't happen.  Those companies who continue to operate with blinders on--milking their decreasing revenue streams as the new world of online applications passes them by--will soon find themselves unable to catch up when they finally come face to face with the new reality.

The next few years will be very interesting.

I've mentioned a time or two before that I don't understand how some web page designers pick their color schemes.  Red text on a black background--which I see distressingly often--is almost impossible to read.  Some people seem to take perverse enjoyment in selecting color combinations that are painful to the eye.  I, on the other hand, know that I'm terrible at picking colors, so I tend to stay with simple defaults like black text on a white background, with blue for highlighting.  I'm more about content, anyway, so as long as my site is readable, I don't need too much flashy stuff.

Except when I do.  If I'm building a site or an application that requires some visual appeal I find myself frustrated at my incompetence.  Picking from standard colors or trying to make my own is hair tearing and mind numbing work.  I usually throw up my hands in frustration and go try to steal somebody else's color scheme.

What surprises me is that there doesn't appear to be a program that will generate a good color scheme for you.  It seems like you should be able to give a program some basic information about preferred background and foreground colors, and it could generate colors for links, visited links, headings, text, and all the other things that normal web pages have.  It should even be able to generate text fonts and styles, colors for menu bars--everything.

The idea of such an application is not far-fetched.  Color matching is quantifiable.  We know how colors appear to the human eye and that certain ranges of colors clash with or complement others.  And there are countless visually appealing layouts from which to choose.  It should be almost trivial to provide a theme selector and a couple of sliders that would vary the different colors in unison and preventing the color scheme from straying too far into ugly land.  Of course, such a program would require some way for the user to set specific colors for individual items, but most users wouldn't need it.

I could envision a slightly smarter version of that program analyzing the colors in a company logo image and automatically generating a complete color scheme.  Why not?

Absent a program to generate colors for me, I've recently found that creating visually appealing color sets is much easier using Hue, Saturation, and Brightness (HSB) than with the old standby Red, Green, Blue (RGB).  The reason is simple:  HSB is much closer to the way that the human eye sees color.  RGB is very close to the way that computer hardware represents color.  With HSB, a slight change in any one of the values results in a slight change to our perception of the value.  This is not the case with RGB, where a small change to one value can have a very large and quite mystifying visual effect on the on the color.

If you're interested, check out John Shemitz's article, Color Theory for Developers, which is a brief but reasonably thorough introduction to the topic.

I might experiment a bit with creating such a program as I've described.  The required information is out there waiting for somebody to gather together.  I wouldn't get rich with it, but it'd be a nice thing to give away.  Perhaps it would save me some eye strain.

I often joke that there are only two things that I do well in the kitchen, and one of them is the dishes.  I usually let people assume that the "other" thing is unmentionable, but if they ask I'll tell them that I make beer.  It's not that I can't cook so much as that I don't particularly enjoy it and therefore don't spend the time necessary to get very good at it.  I can follow a simple recipe.  I've had some failures in the kitchen but usually my food is edible, if a bit on the simple side.

I like pancakes for breakfast on the weekends, and lately I've had the urge to make them myself.  Last weekend I used Bisquik mix, but then got to thinking about what goes into the mix.  So this morning I searched out some "made from scratch" pancake batters and mixed up a batch.  With a few minor variations, the scratch pancake mix contains:

1 cup flour
1 cup milk
2 tsp. baking powder
1/2 tsp. vanilla extract (optional?)
1 egg (optional)
1/2 tsp. salt (optional)
1 tbsp sugar (optional)

This wasn't much more difficult than the Bisquik recipe that called for mix, egg, and milk, and I wonder why people go to the extra expense of buying the pre-mixed powders.  But I digress.

I used the egg and the vanilla extract and dispensed with the salt and sugar this time around.  The pancakes came out okay, although they were a little heavier than I like.  Reviewing the recipe and my ingredients, I realized that I used baking soda rather than baking powder.  That happened because some of the recipes that I'd reviewed called for soda and, because I don't bake, I've never internalized the difference.

Baking soda is bicarbonate of soda, NaHCO₃.  When combined with an acidic ingredient it releases carbon dioxide which makes bubbles in the batter.  When heated, these bubbles expand and lighten the final product.  Baking powder is just baking soda with the acidic ingredient already added in powdered form.  So if you're out of baking powder, you can approximate it by using baking soda and slightly increasing the amount of an acidic ingredient.  More info on this is available at Baking Soda vs. Baking Powder (among other places, I'm sure).

What I did was add the soda but no acidic ingredient to help it form bubbles.  The result was heavy pancakes.  This lends credence to the idea that baking powder makes things rise rather than spread, and baking soda does the opposite.  There is some dispute to the validity of that statement, however, as pointed out in Baking Soda vs. Baking Powder Redux.  However, if you read that experiment in light of the entry linked above, all it really shows is that the homemade combination of baking soda, cream of tartar, and cornstartch made things rise slightly more than the pre-packaged baking soda.

My results with the straight baking soda seem to contradict those experimental results.  It makes sense, really.  Without the acidic ingredient, the baking soda seems to do nothing.  A properly designed experiment would use three batches:  a control batch that has neither baking soda nor baking powder, one that has baking soda only, and one that has baking powder.  My suspicion is that the baking soda only batch would not be significantly different from the control batch, and the baking powder pancakes would be much fluffier.  If I get really ambitious, perhaps I'll try that experiment.

In one of the dumbest moves I've seen yet, the Bush administration in its role as incompetent world parent is going to punish unruly child North Korea by taking away its iPod.  Yes, you heard right, we're banning shipments of iPods, plasma televisions, Segway scooters, cognac, Rolex watches, and other luxury items that Kim Jong-Il likes to give as gifts to his 600 or so closest friends.  The theory, I guess, is that denying him these items will somehow bring him around to our way of thinking.  Yahoo has the full AP story here.

I've never raised children, but even I know that taking away an unruly teen's iPod just makes him sullen and apt to lash out.  Remove enough of a kid's luxuries and he'll find other ways to amuse himself.  Kim Jong-Il, like his father before him, doesn't even care that he can't feed his own people.  Why the Bush administration thinks he'll care about not being able to give some trinkets to friends is quite beyond me.

U.S. foreign policy ever since the end of the Cold War has been inconsistent, incoherent, and mostly just plain wrong.  It's like we're trying to be the World's daddy by using those wrong-headed "be your child's friend" parenting techniques from the 80s that nearly turned a whole generation into sniveling, whining, ambitionless brats.  All these little countries that give us trouble know that we have no power.  They know that if we raise our hand to give a spanking, all they have to do is call Child Protective Services (the U.N. or just the ever-powerful world opinion), and the U.S. is severely chastised by the rest of the children.

If we're going to play daddy--and I'm not saying that we should--we need to use a firm hand, tell the rest of the pissant little ankle biters to behave, and use all the tools at our disposal to implement the changes that we want to see.  If we're not going to do that, then we need to back off and let the world handle their own problems.  This pussyfooting around makes us look weak and stupid, and causes way more problems than it solves, if it solves any at all.

In the past 20 years, Thanksgiving for me has always been a four-day weekend. Prior to that I was working in a bank and, since federal banking regulations prohibit a bank from being closed more than three days in a row (something about depositors wanting to get their money, if you can imagine), Friday would find me ensconced in the computer room banging away at the keyboard. I guess that was okay because my wife at the time was working retail and you can bet she had to work on the day after Thanksgiving.

I tried the "Black Friday" shopping thing once and decided that it wasn't for me. Shopping makes me jittery on the best of days. The crowds on the Friday after Thanksgiving scare the heck out of me. Can you believe stores were opening at five in the morning yesterday? What kind of craziness is that? I can just imagine the employees and customers at each others' throats because they didn't get enough sleep and haven't had enough coffee. Most people I know don't even know that 5:00 AM exists. If they do, they're convinced that there isn't any air in the world that early.

For me, the Friday after Thanksgiving is a day of rest. An entire "me" day for lazing around, maybe watching a movie on DVD, puttering at the computer, and generally being a wholly unproductive member of society. I get very upset if somebody tries to make me actually do work on that day.

The next two days, though, are usually spent on some home project. One year it was replacing a dishwasher and a few other minor chores. This year I'm hanging doors in a room that we're remodeling. There's a trick to hanging doors--especially the double closet doors that I'm working with (two sets of double doors). I'm hoping that I don't have to hang so many doors that I figure out what the trick is. Both sets of closet doors are up now, as is the hallway door--a 30-inch pre-hung door that was very easy compared to these double door units.

I've done a lot of remodeling in this house over the years, including turning the garage into an office, laundry room, entry way, and pantry. I've come to the conclusion that doing your own home repairs or remodeling saves you money only if you don't count your time. It's taken me a whole day (spread out over two days) to install those two closet doors. Somebody who does this stuff for a living would have had both doors installed and perfect in about three hours, tops. And I'll bet the trim would look better than mine, too.

So why do I install these doors rather than pay somebody? Because sometimes I like doing something completely different. I sit behind a computer all day long, thinking and pounding out code or writing articles. It feels good to get out the tools and build something physical from time to time.

Last month I downloaded Ubuntu version 6.06 (Dapper Drake) for two reasons:  to install a desktop machine for some quick testing, and to build a 'permanent' development server.  The only thing painless about the process was downloading the ISO image.  My HP DVD/CD burner, which had been acting a little flaky, completely crapped out and the built-in DVD ROM/CD burner in my laptop had trouble until I lowered the write speed.

It didn't help that this was the same day I rode home in a downpour and discovered that, although my backpack is water resistant, it is not water proof.  My laptop got wet and I was sure I'd fried it.  I left it to dry overnight and everything seems to work okay.  But I digress.

The Ubuntu Live CD ran fine, but installing from that CD failed after a few input screens for no apparent reason.  I poked around a bit and found the answer:  use the "alternate" install CD, which installs in text mode.  After that, things went very well and I was quite pleasantly surprised by the ease of installation and the uncluttered desktop.  I ran the online software update and was again pleasantly surprised at how painless the update was.  We did our desktop testing quickly and then it was time to build the server.

Things went poorly from the start.  I carried the computer back to my office and plugged it in to my spare monitor, booted the alternate CD and told it to install the server.  Yes, I could have gone through the effort of adding server stuff to the desktop installation and removing stuff I didn't need, but it just seemed cleaner to start over.  But after a few screens of data gathering, the monitor went blank.  I don't know why. 

I finally figured out that it was the combination of video card and monitor that was causing the problem.  Why Ubuntu needs to change the video mode in the middle of a text install--especially when building a server that doesn't have a GUI--is beyond me, but there it is.  I've since installed a second server on a different computer that uses that monitor with a different video card.

Finishing the server install doesn't give you a very functional server, and for some reason the default install makes it difficult to get some packages.  Probably for security reasons.  After struggling a while, I found that I had to modify the /etc/app/sources.list file to include the 'universe' and 'multiverse' repositories--something that was not very well documented a month ago, but is explained very clearly now for Ubuntu 6.10 (Edgy Eft).

What software did I have to install manually?  Open SSH, Apache web server, PHP, MySQL, the Java JDK, and the JOE editor because vi sucks rocks.

I'm now in the middle of learning PHP while creating a large Web application.  The server has been very reliable, and I'm beginning to think that I might yet get the hang of this Linux thing--at least as far as servers go.  I still have questions about using Linux on a desktop, but that could change if I can make the time to sit down and play with it.

In June 2004, a team of three people left Vancouver on their bicycles.  They returned almost two years later (720 days), after completing the first human-powered circumnavigation of the Earth.  They cycled, skiied, canoed, hiked, and rowed around the world.  This incredible adventure included, among other things, a completely unsupported five-month rowing across the Atlantic from Portugal to Costa Rica.

One book about the adventure, Beyond the Horizon, will be released in March 2007.  It's on my list. 

I was privilegedl this evening to hear a new member of my Toastmasters club give a speech entitled, "Have you given anybody flowers today?" The title caught my attention, and she kept my attention throughout her five minute talk. I can't recite her presentation from memory, but I did manage to get the gist of it.

When we're young we're taught that "sticks and stones may break my bones but words will never hurt me." It's a nice thought. As we grow older, we learn just how terribly hurtful words can be. There are people who are quite adept at applying verbal sticks and stones. Friends and lovers know all of the most tender spots and can injure with carefully selected words. It's probably not the words that hurt so much, but rather the idea that somebody I love and trust would hurt me on purpose.

If hurtful words are sticks and stones, then what do we call compliments? R (the speaker) put forth the idea that we call them flowers--verbal flowers. Verbal flowers can delight and let another know that you really do like and appreciate him or her. A smile and a sincere, "you look nice today" can brighten somebody's day and maybe help erase the pain of the verbal sticks and stones that we're inundated with on a daily basis. Yes, your partner should know that you love her (or him), but it's so much nicer to hear not only, "I love you," but also, "you look beautiful," or "I enjoy the sound of your voice."

Verbal flowers don't have to be limited just to friends and lovers. Everybody responds positively to compliments. Is it so difficult to say to somebody, "you did a great job" or something similar? In our increasingly critical world, verbal flowers are more welcome than ever.

In the 18 months or so that I've been involved with Toastmasters I've been continually impressed at how quickly a new member can go from literally getting sick at the idea of speaking in front of a group to presenting an idea with aplomb. R. is a beautiful woman in her mid 20s who attended her first Toastmasters meeting just two months ago and was visibly uncomfortable speaking. Tonight she was poised and confident, presenting a wonderful idea with grace, humor, and feeling. It was a joy to watch and listen, and she gave me something that I'll remember forever.

I've been head-down here the last couple of weeks installing a Linux server and trying to get up to speed with some PHP scripting.  I have a lot of notes that I'd like to turn into diary entries, but time to do that is pretty scarce right now.

I ran across an odd one today, though.  My new login script contained this line of HTML:

<td<input type="text" size="32" name="loginUsername" /></td>

That line is obviously broken because the closing brace is missing on the td tag.  The weird thing is that I'd been working with that for a couple of days and didn't notice because Firefox rendered it just fine.  Internet Explorer, when I tested with it this afternoon, failed to show the input box.

I'd make the argument that Firefox is in error here, but seeing as how browsers are unreasonably expected to render horribly broken HTML, there are probably very vocal people who would disagree with me.  I can't help but wonder how much more robust and how much faster Web browsers would be if we could somehow force Web authors to create well-formed HTML.

More as I learn a bit about this stuff.

On October 5, I wrote a short review of Dean Karnazes' book Ultramarathon Man.  In my review I said that Karnazes does a better job than most of answering the question, "Why?"  I also said that I don't have an answer that's any better than his.

In his October 15 entry, On not knowing why, Brook M said:

It seems as if some of these folks do extraordinarily difficult and dangerous things, but don't know why.  At the very least, they can't articulate whatever reasons they might have.  I've always thought one should know why one was doing something.

Furthermore, in response to my saying that most people don't react well to, "Go out and run 26 miles and then you'll have your own answer," Brook writes:

I'm one of those people.  My opinion is that if you can't articulate the reason for an action, then you don't have a reason.  You're simply working off instinct.

If nothing else, 12 hours alone on a bicycle gives one a lot of time to think.  And I thought a lot about why I was spending an entire day pedaling when most people would have spent the morning recovering from the previous night's birthday bash.  Somewhere along the way I reached a conclusion.

I have a whole host of reasons why I choose to participate in endurance events.  I enjoy being outdoors.  I exercise in part because it keeps me healthy.  Being out on the road bicycling, or on the trails running, allows me to clear my mind and release some of the tension from the day.  I like feeling tired but somehow refreshed after a ride.  I enjoy talking with other people I meet on the road.  I like being able to eat that extra piece of pumpkin cheesecake without worrying about its fat or caloric content.  I like pushing my body to see how far and how fast I can go, always striving to improve.  And I get a real kick out of watching the expression on somebody's face when I tell him that I rode 160 miles on my bicycle in 12 hours.

Those are my reasons and it's quite likely that people who don't participate in endurance events won't accept them.  But that's okay.  I don't understand why somebody would spend all of Sunday afternoon sitting on the couch watching football and drinking beer.  For that matter, I don't understand why anybody would spend much time at all in front of the TV.  I have no desire to own a boat or to spend all day standing in hip-deep water, flicking a line and hoping some fish will bite.

People who drive trucks for a living probably don't understand why I like sitting in front of the keyboard banging out code.  That's okay, because I don't understand the attraction of driving all day.  The only real issue is that endurance events are so far out of the realm of common experience that most people look askance at those of us who participate.  Almost every endurnace athlete will tell you that he anticipates and even enjoys getting those reactions.

You have your reasons for the things you do, and I have my reasons for the many activities I enjoy.  We might not accept the other's reasons as valid for us, but as big-brained primates, hopefully the smartest species on the planet, I think we can accept that the other's reasons are valid for him, and agree to get along even though we don't fully understand each other.

I rolled out at 6:04 AM yesterday, in an effort to see how much ground I could cover in 12 hours.  It was 45 degrees outside, so I was wearing my arm and leg warmers, a hat to keep my ears warm, and gloves.  The first two hours--especially the second hour--were very cold as I headed north to Lake Georgetown and then west to Liberty Hill.

I had packed my camera along in the hope of getting pictures of myself at points along the route.  The only pictures I managed were at the two spots along the route where Debra met me, and this one excellent shot of sunrise a little after 7:00.

The sunrise was incredibly beautiful, making the frozen fingers almost irrelevant.  I soon turned west, though, putting the sun behind me as I pedaled towards Liberty Hill.

I've done quite a bit of long-distance bicycling over the last few years and never encountered the problem I experienced during the first four or five hours of today's ride.  I couldn't pee enough.  I know that sounds odd, but there it is.  I'd stop to empty my bladder and five or ten minutes later I'd have to go again.  Riding with a full bladder is very uncomfortable, and it was frustrating having to stop every 20 or 30 minutes.  At my average moving speed of about 16 MPH, every minute off the bike is a quarter mile.  Sure, I need some rest, but the idea behind the ride was to see how much ground I could cover in 12 hours.

Outside my favorite convenience store in Liberty Hill, I met three riders who were just starting a 60-mile ride.  We rode together for 30 minutes or so before they turned off and I headed south on Bagdad road to meet Debra at the 40-mile mark.  The primary purpose of that meeting was for her to take my lights and some of the cold weather gear.  She brought Charlie along and got this picture of us just before I rolled out.

I was feeling very good except for the having to pee every 30 minutes part.  I'd been keeping my heart rate down, and was eating and drinking regularly in order to stay hydrated and keep my energy up.  The next part of the route--from the Walgreens in Cedar Park to the Veloway in south Austin--contained most of the day's hills.  I wove my way down to highway 620 and headed south past Mansfield Dam through Lakeway and then Bee Cave Road back into Austin.  I met a couple of other riders along the way, sharing the road and a few moments' conversation before they turned off on whatever route they were taking.

I got hungry--ravenous--at about 70 miles.  I stopped at a convenience store to refill the liquids, and scarfed down some munchies.  I also called Debra and asked her to bring some peanut butter and jelly sandwiches to our next planned meeting place.  I don't know what it is, exactly, about PB&J that I like so much during a bike ride, but those sandwiches really hit the spot.  I was so hungry she had to tell me to slow down, I was shoveling it in so fast.  It was about 1:00 PM and I'd covered 93 miles.  Other than being hungry, I was feeling really good.  No muscle aches, and surprisingly my butt wasn't sore from sitting in the saddle all that time.  I unloaded the rest of my cold weather gear, Debra snapped a picture, and I was off again, heading north back through Austin and towards home.

Long distance cycling can be a very lonely sport.  It's not for those who are uncomfortable keeping themselves entertained.  It's true that I have to keep some attention on the road and be aware of what people in their cars are doing, but that becomes almost second nature after a while.  One must find a way to occupy the mind or it starts to occupy itself.  My brain seems to like putting particular annoying or catchy songs in a tight loop unless I'm careful to think about something else.  I can't solve very involved problems while I'm riding, but I can toss ideas around.  And doing so keeps my mind from contemplating the craziness of what I'm doing.

I met my friend Shelia (Jason's wife) at about 105 miles, and we stopped at their house for a break.  She offered me a protein bar which I didn't think I needed, but I accepted it anyway.  How wrong I was!  About 20 minutes after eating that I was feeling a whole lot better than I had been.  One of the dangers riding for so long is that you tend to discount the pain and often don't recognize when your body is about to crash.  That protein bar and a bottle of Gatorade (about 500 calories, all told) did wonders, and I was able to pick up the pace a bit.

Sheila rode with me for about 20 miles before she had to head back home.  I headed north towards Round Rock, stopping at a convenience store for a hot dog and a Coke.  It was 4:00 in the afternoon and I was looking forward to finishing the ride.

Since I was feeling so good, I picked up the pace a bit as I rode through the neighborhoods near the house.  It's amazing how much distance you can cover without really going anywhere.  Knowing the neighborhoods as well as I do, I was able to time things just right so that I pulled into my driveway right at 6:04 PM, exactly 12 hours after leaving.  I'd covered 160 miles, with a total of ten hours and 13 minutes pedaling.  The other hour and forty seven minutes were spent on rest stops and traffic lights.  My body was tired, but I wasn't completely exhausted.  No major pains or muscle aches, and I was fully coherent.

What surprises me most about the ride is that I hadn't done many long rides over the summer.  Most of my 'training' has been the seven mile commute to work and back each day, with a few rides of 50 to 70 miles on the weekends.  But I hadn't been focusing on the training as much as I had in the past.  I knew, however, that if I kept my heart rate down in the lower part of my areobic range, and kept fed and hydrated, that I could go essentially forever.  At least, that was my theory.  And it turned out to be correct.  I've established a baseline.  I know that I can do 160 miles in 12 hours.  The next time I'll do a little better.

Yesterday was my 45th birthday.  As some of you might recall, I had planned a birthday challenge, but then backed out of it due to otime pressures and my injured shoulder.  But I'm not going to be a complete couch potato.  It is now 5:11 AM on Saturday the 28th.  I'll be rolling out at 6:00 for a 12-hour bicycle ride.  I'm hoping to cover at least 150 miles.

I don't have the technology to post while on the road.  I'll update this post when I finish, and then add detail tomorrow.  Provided I'm still functional.

Two weeks ago I asked, "Who was Herm Albright?"  I've had a few people tell me that they searched and couldn't find anything.  I finally ran across an entry for Herman Oliver Albright at AskART.

He was a painter and lithographer, born on January 29, 1876 in Mannheim, Germany.  He emmigrated to the United States and settled in San Francisco in 1905.  He held a job with the Paul Elder Book Company for 25 years  He died on September 21, 1944.

I have been unable to find any detailed biography of Herm Albright or any information about his quotes.  I'm not entirely sure that this is the same Herm Albright, although the birth and death years in the artist's bio match up with the years cited in some of the quotes attributed to him.

Again, if you have any information you'd like to share with me, I'd be grateful.

I want to like Linux.  Really.  But every single time I start working with it I end up so frustrated I wonder why I don't just give up.  We're going to need server support for our new project and, all things considered, Linux looks like the way to go.  I've been hearing such good things about Ubuntu over the past year that I thought I'd start there.  Following is the result of two days' fiddling.

Download Ubuntu Desktop first, because we wanted to test something.  Download went okay.  Then I had a heck of a time burning the CD.  First my relatively new DVD/CD burner checked out, and then the built-in on my laptop wouldn't work at high speed.  I made a half dozen coasters before I got a good burn.

Ubuntuu booted okay from the live CD, but it wouldn't get past the "select language" on the install.  It'd just hang.

I spent a couple of hours searching for the reason, and stumbled across a few others who were having similar problems.  The solution was to download the alternate install CD and do the install from there.  I got the image last night and burned it to CD at home.  (Another story I'll relate in a different post.)

This morning the Ubuntu install went fine.  I was impressed with the clean desktop.  The default GNOME GUI looks very nice.  The software update process went without a hitch--very nice.  I was able to download the Flash 9 update for Linux.  We confirmed that worked, and I also got the Flex SDK running on the machine.

The machine was pretty sluggish with only 128 MB  of RAM, so I made a trip to Fry's for a gigabyte.  Came back, installed the RAM, checked to see that the machine still worked.

With that out of the way, it was time to install the server.

Moved computer back to my office, hooked everything up, and started server install in text mode.  It'd get to a point and then ... blank screen.  Why?  Sure, I'd hooked up a different monitor, keyboard, and mouse, but that shouldn't affect anything.  Right?

After fiddling around and trying different things, I carted the other monitor back into my office and hooked it up.  Works fine.  I can't figure this one out.  Why does the old Planar LCD panel work and my Viewsonic VP151 not work?  Imagine if I didn't have another monitor and tried to install Ubuntu.  I'd just give up and throw the CD in the trash.

With that problem solved, I got the server installed.  "apt-get update" worked nicely to get all the latest updates.

Now to copy a few things from the network.  How?  No Samba support in the default install.  Okay, so I install that (apt-get install samba, as pointed out in the Ubuntu documentation) and try to mount the network drive.  No dice.  Create a mount point and try again.  Still no dice and I get a nice informative error message:  "mount_data version 1919251317 is not supported."  WTF?  A few Google searches and I found that all I needed was to install smbfs.  "apt-get install smbfs".  Would have been nice if the Ubuntu documentation that talks about installing Samba had mentioned that.

The Flex SDK comes in a zip file.  No, not a gzip, but a PKZIP format.  Fortunately, I was able to install unzip:  "apt-get install unzip"

The first thing to try when you install the Flex SDK is building the examples.  But now I'm getting error messages saying that the "java" command is not found.  Great.  "apt-get install java" didn't do it, so back to the Web to figure out what package I need.  Finally find "sun-java5-jdk".  But that package isn't found.  More searching and I find that I need to update my /etc/apt/sources.list file as described here (http://ubuntuguide.org/wiki/Dapper#How_to_add_extra_repositories)

And that's where I left it until tomorrow morning.

Frustration.  Take a step, get stumped.  Find something.  Take another step.  Lather, rinse, repeat.  I figure at this pace I'll have a reasonable server install sometime around Christmas.

 

I found more info a couple of weeks later.  Click to view.

It appears that Herm Albright's most famous quote is, "A positive attitude may not solve all your problems, but it will annoy enough people to make it worth the effort."  It's a funny quote, with several possible meanings.

A friend of mine posted that quote on her blog and then asked, "Who is Herm Albright?"

From what I've been able to determine, Herm Albright lived from 1876 to 1944, and he might have been a writer for the Saturday Evening Post.  That's all I've been able to dig up, and I'm really curious.  Wikipedia doesn't have an entry, and all the search engines turn up quote after quote, but nothing about the man or his life.

Anybody have old copies of the Saturday Evening Post?  How about a little author bio?  Anything would help.

I just finished reading Dean Karnazes' book Ultramarathon Man, in which he describes how he became an ultramarathoner and attempts to answer the question that everybody asks: Why?  What makes a person want to abuse his body by running for two days straight, eating on the go, suffering the wild mood swings from euphoria to deep depression and back again, and enduring the aches, cramps, blisters, exhaustion, and everything else that goes along with ultraendurnace events?

Karnazes comes closer to an answer than most, I guess, but he freely admits that even he hasn't quite figured out why he does it.  I could sum up his reasons here, but they wouldn't make much sense unless you've read the 275 pages that make up the context of the answer.

When I'm asked why I ride 100 miles or more on my bicycle, I'm tempted to give the glib responses, such as George Mallory's, "Because it's there" response to why he wanted to climb Mount Everest, or my own favorite, "Because I can."  But in truth I don't have any better answer than Karnazes, even though I've thought about it quite a bit.  The best answer I've seen is more of a challenge: "Go out and run 26 miles and then you'll have your own answer."  Not surprisingly, most people don't react well to that.

Karnazes writes with an engaging conversational style and inserts a lot of the somewhat offbeat humor that I find common among endurance athletes.  He does a very good job of pulling the reader into the story--trying to explain the ups and downs, pain and enjoyment of competing in an hours-long event.  I might be better equipped than most to appreciate it, having done some endurance events myself, but I think anybody who enjoys reading about great adventures will enjoy the book.

Highly recommended.  As of today, you can still get the bargain price of $4.99 from amazon.com.  It's a delightful read and well worth the price.

When I was seven years old, my dad moved us to a house that had a swimming pool. We five children, ages four to ten when we moved in, spent the next five or six summers in the back yard, swimming in the pool, jumping on the trampoline, sunning ourselves, or running around like a pack of yard apes as kids are wont to do.  It was a wonderful way to grow up.

The back yard also contained a very large covered patio with two huge (hey, I was a little kid) picnic tables, some lounge chairs, and an outdoor fireplace where we'd sometimes roast marshmallows at night. Every afternoon, Mom would make a plate of sandwiches and bring them out to the patio, setting them on the picnic table and calling us and whatever friends were there to sit down and eat.

I'd sit down prepared to dig in and Mom would say, "Jim, where is your shirt? You know you can't eat lunch without your shirt on." Whenever we sat at that picnic table, we wore shirts. Even my sisters had to put on shirts to cover their bathing suits before they were allowed to sit and eat.

Almost 40 years later, at my own house now and making lunch to eat out by the pool, I sit down at the table and hear my mother's voice in my ear: "Jim, where is your shirt?" Honestly, I get up to find a shirt, even if that entails walking back into the house and digging one out of the dresser drawer. I think I just might experience a mental break were I forced to eat lunch without my shirt on.

I think others have similar tales: times when they hear a parent's voice in their heads, or rituals that they engage in due to upbringing.  We all have them: the way we tie our shoes, the food we call "comfort food," the way we brush our teeth or groom ourselves. The shirt at the dinner table thing is one of my favorites to relate because it shows how seemingly insignificant things from our upbringing affect our lives in so many ways.

Anybody else have random things like that? Things that you don't even think about, but strike other people as odd?

Part of my daily bicycle commute has me riding about a half mile on a sidewalk through a neighborhood association's greenbelt.  The sidewalk is meant for bicycles as well as pedestrians and it shaves about a mile off my ride.  More importantly, it keeps me off a busy two-lane road that has no shoulder.

Most days I encounter people walking on the trail.  If I approach from the front, of course, it's rare that I need to attract anybody's attention.  I slow down and make eye contact, though, to be sure that people really do see me coming by.  I smile and say hello, and then continue on my way.  But when I approach from behind I have to warn people that I'm coming by.  Not only as common courtesy, but also as a safety measure.  Many people are walking dogs or walking with children, neither of which I'm particularly interested in colliding with.

As I approach from behind, I get within range where I can say "bicycle behind you" loud enough to be heard, but not so loudly that I'm screaming.  I always slow down, of course, and if the person does not acknowledge, I will repeat it, "bicycle behind you", in a louder voice and much closer.  Most people hear the first time.  Almost all hear the second.  Without fail, they thank me for making my presence known and they ensure that there's room for me to pass.

Periodically I come upon somebody who's listening to music or somehow otherwise so preoccupied that he (or she) doesn't hear my warning.  Since I don't feel like screaming to scare anybody, I just ride slowly by and hope that he doesn't take that moment to start his Fred Astaire routine.  All good, right?  Except that almost every time I do this, the person with the music blaring in his ear will yell obscenities about rude bicyclists.  It's absurd.  I'm in too much of a hurry and too non-confrontational in any case to turn around and explain that I made my presence known but he was too involved in his music to hear it.  Probably best in any case, as it's doubtful that such actions would have a positive effect.

There's a hike and bike trail near the house where I ride my mountain bike from time to time.  I perform the same ritual there and experience the same results.  Conversations with other bicyclists have confirmed that this is a common problem.  We do what courtesy requires and yet we're verbally abused because some people don't have the common sense to turn down their music so that they can hear what's going on around them.  What, exactly, are these people thinking?

Yes, I've been a little busy and haven't taken the time to keep up here.  Perhaps later in the week I can spend some time on it.

• The third and last article in my series about extending the .NET console interface, Console Input in .NET, is available on DevSource.  You can download the code for the articles at http://www.mischel.com/consoledotnet/consoledotnet.zip.
• Some English word usage still confounds me.  Today I was trying to figure out whether I should use "lie", "lay", or "laid" in a sentence, and I found Grammar for Geeks: Lie vs. Lay vs. Laid.  Problem solved.
• The main Grammar for Geeks site has links to several similar topics such as There vs. Their vs. They're and others.  I especially liked the Words that are not real. 
• Speaking of non-words, I did a Google search for the non-word "permanentize," which I ranted about a few years back.  I was surprised to see that my blog entry was the top search hit.  Maybe I shouldn't be excited about having the #1 site  for a non-word?

Dean Karnazes is attempting to run 50 marations in 50 consecutive days, in 50 states.  He started on Sunday the 17th.  Today he's in Little Rock, Arkansas.  He calls the event The North Face Endurance 50.  He also maintains a blog, Endurance Is.  This is some seriously impressive craziness.  I've run marathons, and I can tell you first hand that the last thing most runners want to do the day after a marathon is get up and run another one.

In the world of ultramarathoners, Dean Karnazes stands out.  Men's Fitness said he "might just be the fittest man in the world."  He regularly does 100 mile ultramarathons, and once ran 350 miles non-stop:  foregoing sleep for three consecutive days.  He's a mountain biker, surfer, triathlete, and I don't know what all else.  His book Ultramarathon Man is an international best seller.  Most people look at what this guy does and just shake their heads over the insanity of it all.  Not me.  I know what kind of dedication it takes to do what he does.  I'm impressed by anybody who sets a goal and continually works toward it.

I'm also quite impressed by the Endurance 50 web site.  This an an excellent use of Flash, and the site is very well designed, pleasing to the eye, and easy to navigate.  It displays the necessary information in an attractive design, without any extraneous bells and whistles.  It's a pleasant departure from all too many sites that look as if they were designed by people who think, "more is better."

I spent some time over the weekend adding a Title Index to the blog here, and also rearranging the monthly archive pages.  In the past, the monthly archives had URLs of the form /diary/year/year_month.htm.  That is, the archive for October 2005 was http://www.mischel.com/diary/2005/2005_10.htm.  I don't remember how I came up with that convention, but it's kind of wonky.  I've changed things now so that the URLs are of the form /diary/year/month/index.htm, so that if you want to see entries for October 2005, you can go to http://www.mischel.com/diary/2005/10.  I've created redirect commands so that the existing old-format URLs will work, but there won't be any old format URLs for new months.

I suspect that the Title Index will be useful mostly to me.  In almost six years of blogging I've covered a lot of different topics, and I find myself referring back to them quite often.  I've been using Google to search the site, but I think the Title Index will be much easier and it will prevent me having to filter the Google search results when I know that what I'm searching for is in the entry's title.

I've said before that I'd like to move to a different blogging program.  The problem is finding the time to convert everything, and I dislike the idea of trying to keep two blogs in sync for even a short period.  I'm pretty sure now that I won't be installing WordPress or any other blogging sofware on my server.  I am, however, considering either moving the blog to one of the blogging sites, or switching to a provider who includes blogging software as part of the hosting package.

I was sitting in the conference hall this afternoon, working on an article and listening to the music they were playing while waiting for the next session to start.  I can block out most music if it's not too loud or incredibly offensive, but when something out of the ordinary comes on I get shocked out of my zone.  Such a shock came when what sounded like Mariachi music started coming from the speakers along with lyrics in Spanish.

Then I laughed out loud when the language changed to English, singing one of my favorite wacky songs.  This rendition is different, though:  Flaco Jimenez singing En El Ceilo No Hay Cerveza (In Heaven There Is No Beer) in Spanish, English, and Dutch.  Definitely laugh-worthy to me, and enough to get me to buy his latest CD.

I'm at the Flashforward2006 conference in Austin today, tomorrow, and Thursday.  I honestly don't know how many people are here, but it's a lot.  I heard somebody say 300, but I think there were at least 500 in the audience at the keynote.

It's impossible to get an unbiased view of a product or technology when you're at a conference that's specifically about that product or technology.  Almost everybody here has a vested interest in the success of Flash and wouldn't be here if they didn't think there was a very bright future for it.  Still, there are many things to indicate that Flash will only grow in popularity.

At the keynote, Kevin Lynch, Chief Software Architect at Adobe and formerly a major player in the development of Flash for Macromedia, gave us some interesting statistics on the adoption of Flash.  Version 7, according to Lynch's numbers, was adopted at the 70% level in about 12 months.  Flash 8 reached 80% adoption within 9 months of its release.  For Flash 9, which was released less than two months ago, Adobe is projecting a 50% adoption rate in three months, with 80% adoption in six months or less.  With major sites like MySpace and YouTube already including Flash 9 content (YouTube has standardized on Flash 9), those projections are probably realistic.

From my perspective, the key new features in Flash 9 are the new runtime and the JIT compiler that converts Flash bytecode to native code.  The result is that Flash 9 applications have much better performance than previous versions.  During the keynote today they demonstrated a particle animation effect in Flash 8 that was trying to run at 33 frames per second.  Computing the particle effect took a little more than 100 milliseconds per frame, so their real frame rate was something less than 10 fps.  The same application compiled for Flash 9 takes about 15 ms per frame, giving them plenty of time for other things while still maintaining 33 fps.  Flash 9 doesn't have the performance of .NET, but it probably could.  Right now, there's probably a 10x performace hit running Flash (that is, for CPU-intensive applications, a native application will run 10 times as fast as a Flash application), but that's still plenty fast.  Flash 9 applications run faster today than native applications ran in 2000.  One can only assume that Adobe is working to improve the ActionScript compiler and Flash player's performance.

With Flash 9 we have a cross-platform (they demonstrated the Linux version at the keynote) virtual machine that can play video and audio, and perform animations.  With the new Flex 2 framework, it's possible to create real interactive user interface applications that are richer and perform better than equivalent Ajax applications.  And the best thing is that, for all practical purposes, every Internet user has the Flash player or can get it painlessly with a simple download.  Microsoft even ships an early version of Flash with Windows XP, and the Express Install feature will upgrade the Flash player seamlessly.

About the business of Flash, I'm less sure.  The people here do seem to be excited about the future, and almost all of the presenters have mentioned that their companies are hiring Flash developers (artists as well as ActionScript programmers).  It's possible that this is all a lot of hype reminiscent of the late '90s Internet boom, but it doesn't feel like that.  I honestly believe that there is a lot of room for innovation in Flash and a lot of opportunity for small developers.  It will be interesting to see, three or four years from now, whether I'm right.

The Flash application framework, MX, provides a powerful and easy to use platform for creating user interface applications.  In many ways it's similar to the .NET Framework, and similar packages that I've used in the past.  However, there are key differences that will surprise a newcomer.  I ran into one this morning when I wanted to create a modal dialog box that would gather information and return it to the caller.

When you create a modal dialog box in Windows, execution of your main program is halted until the dialog box is closed.  The simplest way to illustrate this is with a message box:

public int SomeFunction()
{
    MessageBox.Show("Hello, world");
    // next statement will not be executed
    // until the messagebox is dismissed.
    return 1;
}

When the message box is displayed, execution is transferred to the code that handles the message box and the user cannot interact with the main program.  The same holds true for modal dialog boxes.  In Windows.

Under Flash, things are a bit different.  As with Windows, a modal dialog box in Flash prevents the user from interacting with the window or form that spawned the dialog box and execution of the main program is suspended until the user dismisses the dialog box.  The difference is when that suspension takes place.  Under Windows, it's immediate.  In Flash, program execution continues until your program returns control to the Flash virtual machine.  Consider this method that is executed when the user presses the "New Game" button:

    private function startNewGame() : void
    {
        var dlg : NewGameDialog;
        dlg = NewGameDialog(PopUpManager.createPopUp(this, NewGameDialog, true));
        // do some stuff
        return 1;
    }

In this function, execution continues in the function after the popup is created.  Flash displays the dialog and disables the caller's user interface until after the dialog box is dismissed.

What does it mean?  A program can't just display a dialog box and in the next line of code get the results from that dialog box.  Instead, the program has to display the dialog box, return, and then wait for a message or event to indicate that the dialog box has been dismissed so that the program can harvest the results.  It's a much different programming model than Windows.

I'm not saying that either way is better than the other--just that Flash is different from other systems and it's going to take some time getting used to the new model.

Over the past two months of almost daily bicycle commuting I've made a few observations about the reactions of drivers to bicyclists, and also learned a few things to watch out for.  Fortunately, none of those lessons have involved me hitting the pavement.

• Most drivers are friendly enough and will go out of their way to make room for me:  letting me pass in front of a turn lane, and moving far to the left side of the lane when they pass.  I don't let that lull me into a false sense of security, though.  I'm careful to make eye contact and telegraph my intentions before putting myself in front of a moving car.  It only takes one mistake to become road pizza.
• In general, drivers are much more courteous during the morning commute than they are in the evenings.  The worst time to be cycling on the road is between about 4:30 pm and 6:00 pm.  Drivers seem to be in a very big hurry to get home, and they'll do stupid things like cut me off in a turn lane just to save a second or two.  Friday evenings are especially bad.
• There's one exception to the above rule.  Drivers get more anxious and much less indulgent as they get closer to their destinations.  My route to work takes me by a large office complex into which many cars turn each morning.  Because it's near the bottom of a pretty good hill, I'm usually moving at a pretty good clip--30 MPH or so--when I go through that intersection.  I position myself at the left edge of the right turn lane rather than on the shoulder so that I don't get hit with a right hook (scroll down to tip #4) by somebody making a right turn.  That's all well and good until some idiot whips past me at 40 MPH, swerves into the turn lane, and then slams on his brakes to slow down for the right turn.  I pass by on the left two seconds later, thinking not-nice thoughts about the driver.  Next time I'm going to rap on the window of the car as I go whizzing by.
• Although it's fun to zip along on the shoulder at 20 MPH, passing a miles-long line of cars stuck in traffic, it can be dangerous.  Not having to worry about an inattentive driver plowing into me from behind, it's tempting to let my guard down and crank away unaware.  This morning that almost cost me.  I looked up and saw a car making a right turn into traffic--right in front of me.  Traffic on the road had stopped to let this guy in, but he hadn't seen me barreling toward him.  I'm glad I have good brakes.
• Briefly stated, The Law of Gross Tonnage says, "thou shalt not argue with anybody whose vehicle outmasses your own."  In nautical circles it's a matter of physics: a larger craft is much less maneuverable than a smaller craft.  When applied to the world of cars and bicycles, it's more a matter of common sense.  I'm not about to try to insist that a driver yield to me.  Although many would yield, it only takes one inattentive or malicious driver to ruin my day.

We can complain all we want about drivers being clueless or even discourteous, but that's not going to change.  As much as I wish that drivers would pay closer attention and look out for bicycles, it just doesn't happen.  If you're on your bicycle sharing the road with cars, it's in your best interest to assume that the drivers don't see you, to always make eye contact before putting yourself in harm's way, and always leave yourself an out.

In my first games programming job, I took over development of a game that the original designer had started.  He had written a "first playable" version that included basic game play and graphics.  My task was to work with the publisher to complete the design, and then implement that design.  The contract called for a Windows version and a version for the Macintosh.  Since we didn't have any in-house Macintosh experience, we contracted with another development firm to complete the Mac version.

The entire project was written in C.  We wrote all of the internal game logic to be platform independent so that it could compile without change on either platform.  I put all of the Windows-specific user interface code in separate modules, and had a very strict separation between platform-specific and platform-independent code.  Such was common practice, and it had worked well for us in other projects that we had developed for multiple platforms.  All the other programmer had to do was develop the Mac-specific user interface and attach it to the base game logic.  Simple, right?

The first few milestones in the Mac port came and went, with the programmer showing good progress on the port.  One day, however, we got a call from the owner of the other development house.  He had to fire the programmer who had been working on the game.  Why?  Because the programmer--a very junior developer who was working at his first job--had decided that it would be easier to throw out the 20,000 lines of fully-tested platform-independent C code and replace it with entirely new code written in Forth.  Let's just say that none of us involved in the project were very happy about that.

I don't have anything against Forth as a programming language, and I'll be the first to admit that my C code didn't present the best possible interface, but discarding a large body of existing and proven code in favor of something new is almost always a bad idea.

This "throw it away and start over" mentality is very common among junior developers who don't understand that things are usually much more difficult than they appear.  They think that it's easier and faster to rewrite code from scratch than to take the time to study and understand the existing interfaces.  Besides, implementing user interface code isn't nearly as cool or interesting as developing core game logic.  And we all know how programmers will favor cool and interesting over tedious UI hacking.

Ultimately, of course, the blame lies squarely on the project manager who let several milestones go by before performing even a cursory inspection of the code.  Had he examined the programmer's work at the first milestone, it would have saved him a whole lot of time, money, and heartache.  He had to pull a programmer from another project so that they could finish our contract in time.

The moral of the story is twofold.  First, a complete rewrite takes longer than you think, especially if you don't fully understand what you're rewriting.  Not only that, the new code probably has a large number of non-obvious bugs that probably existed in the old code, but were fixed over time.  The old code, as clunky as it might appear, contains a whole bunch of hard-won knowledge that will almost certainly be lost in a rewrite.

Secondly, programmers need supervision, especially junior programmers at the beginning of a product development cycle.  You have to live with your early decisions throughout the product cycle, and any mistake you make early on will be very expensive to fix as time goes by.  It's imperative that managers keep an eye on the development team's early decisions.

I guess it's a good thing that I've been too busy lately to find little filler snippets for my Odds 'n Ends box.  Just a couple of notes today.

• As an exercise in learning more about the new Flash 9, David and I have built an anagram/crossword finder.  It's been an interesting experience, and has required us to learning a surprising amount about ActionScript and the internal workings of the Flash runtime system.  If you play with it, I'd appreciate any feedback.
• The annual Women's Adventure Race will be held in the Austin area next Sunday the 10th.  We can always use volunteers to help out with many aspects of the event.  The Williamson County Amateur Radio Club will be providing communications again.  If you're interested in volunteering to help at the event, contact volunteer@womensrace.com.  If you want to help with communications, contact me by clicking the "contact" link on the left.

The second article in my series about making the full Windows Console API available to .NET programs has just been posted to DevSource.  This article, Working with Console Screen Buffers in .NET, describes screen buffers and their use.  Screen buffers allow you to pre-compose console output in an offscreen buffer and then make that buffer active--effectively redrawing an entire screen in a single operation.  The ConsoleScreenBuffer class that I developed also provides some much more flexible output methods that aren't available in the standard System.Console class.

Full source code for both articles, including many examples in C#, is available here.

Serendipity is an odd thing, isn't it?  Monday I wrote a little rant about how a project needs a strong leader in order to be successful.  I've received some negative feedback on that assertion, but nobody yet has shown me a successful project that doesn't have a strong leader.

Back to the serendipity.  Today I ran across Charles Hannum's letter, The future of NetBSD.  Charles is a founder of the NetBSD project, and has been with the project throughout its life.  He was a pioneer in the development of collaborative--what later became known as "open source"--projects.  He has a lot of things to say about the NetBSD project, most of it not very flattering.  In fact, the first sentence tells you it's not going to be pretty:  "The NetBSD Project has stagnated to the point of irrelevance."

After explaining how the project was structured and how that evolved into a very common structure for open source projects, he says:

Unfortunately, we made some mistakes here.  As we've seen over the years, one of the great successes of Linux was that it had a strong leader, who set goals and directions, and was able to get people to do what he wanted -- or find someone else to do it.  This latter part is also a key element; there was no sense that anyone else "owned" a piece of Linux (although de facto "ownership" has happened in some parts); if you didn't produce, Linus would use someone else's code.  If you wanted people to use your stuff, you had to keep moving.

NetBSD did not have this.  Partly due to lack of people, and partly due to a more corporate mentality, projects were often "locked".  One person would say they were working on a project, and everyone else would be told to refer to them.  Often these projects stagnated, or never progressed at all.  If they did, the motivators were often very slow. As a result, many important projects have moved at a glacial pace, or never materialized at all.

He mentions other problems, but they're all due to lack of leadership.  And his outline for moving the project foreward (a nice idea, in my opinion, but not likely to succeed) involves replacing the current "leadership" of the project with people who are commited to turning it around and making it work.

The letter serves as a good warning to those who would try to start a project in the absence of strong leadership, or decide at some time that the project is stable enough that the strong leader can be dispensed with.  That's almost always a bad idea.

Did you ever wonder why some seemingly simple projects fail and others--often much more complicated--succeed?  It doesn't matter whether it's open source or traditional development, or if the development team is filled with highly experienced programmers or primarily with beginners.  I've seen big projects completed on time and under budget, and small projects drag on indefinitely.  What's the difference?  Adult supervision.

Seriously, programmers in general are terrible at time management.  We like to work on cool stuff.  Without somebody pushing us, we tend to work on a problem until we've figured it out and then move on to the next interesting problem.  That's fun!  Turning something we've figured out into a finished, debugged, tested, and documented unit is hard work and not terribly exciting.  Sure, we like to ship stuff but, like most people, we don't want to do the not-fun stuff.

The "cool stuff" disease also causes programmers to add frivolous features like skins to programs before adding the real functionality that makes the program useful and attracts customers.  I can't count the number of applications that have really cool UI features with customizable everything, but don't do the basic job they're designed for.  Again, the programmers worked on what they wanted to work on rather than concentrating on and completing the important parts of the project.

The job of the project manager is to keep the programmers focused on the final goal--shipping a complete, functional application on time and within budget.  Somebody has to be the Daddy.

The uninformed theory of open source software is that the development teams are this loosely-organized group of individuals who magically work together and do everything by consensus.  That's far from the truth.  I've notedbefore that almost all successful open source projects are controlled by a small group of "core" developers--often just one person--who reviews all issues and issues an edict if the group can't reach a decision.  This is often referred to as the Benevolent Dictator model.

Software development projects aren't the only places where you need a Daddy.  Wikipedia works because there is a social infrastructure that places a trusted person in charge of each topic.  That person doesn't have to rule with a heavy hand, but he does need to make decisions about the content and style of an entry.  Can you imagine the chaos if just anybody was allowed to make changes to a topic with impunity?

Any collaborative effort requires a Daddy.  Somebody has to be in charge.  Without a leader, the effort will fizzle and die, or explode spectacularly into flame wars and hard feelings.  Sure, some people won't like the tight control, but that's okay.  Nobody's irreplacable, and those who can't work under the rules of the game aren't worth trying to bend the rules for.

• I hate waiting in line for things.  Last weekend while Debra and I were waiting for the doors to open for the home show at the convention center, I thought I'd get a little exercise.  (It's a 4-megabyte QuickTime video so it might take some time to download.  No sound.)
• I bought a unicycle on Saturday and am learning to ride it.  I've managed not to hurt myself yet.  I can get about four pedal strokes (approximately 10 feet) before I lose my balance.  Pictures as soon as I get the knack of it.
• Bananagrams will deliver a singing telegram by somebody wearing a banana suit.  Hey, at least it seems to be a going concern.
• Bananagram.com, featuring BanaBot, was going to find the cheapest prices on common grocery items so you could save money by visiting six different stores.  This is funny not for the questionable premise, but for the "news" on the site.  November 2004: launched with high hopes.  December 2004: hire fly-by-night Russian developers to create the site.  June 2005: with a pretty .JPG mock-up and no more money, the failed project is abandoned and left as a warning to everybody else.  Investment opportunities still exist.  Act now!

Okay, so I haven't had a lot of time recently for writing blog entries.

Although graphical user interfaces are all the rage these days (and rightly so!), there's still some call for character-mode (console) applications.  In .NET versions 1.0 and 1.1, support for console applications was limited:  sequential access to the standard I/O streams.  .NET 2.0 introduced a much more complete console interface that allows cursor positioning, setting text attributes, and a few other low-level console tricks, but it doesn't even come close to supporting the full power of the Windows Console API.

My article, Extending the .NET Console Interface, is the first of three planned articles that explore the Windows Console interface and present a .NET class that exposes the entire API to .NET programs.  This first article covers creating and attaching to consoles, responding to console control events, and using console aliases.  The second article will discuss the console screen buffer in detail, and the third will be all about using the console input buffer.  Full source code for the article is available for download here.