Jim's Random Notes - January 2005

I don't know what changed, but I got 33% more unique visitors to my site this month than last month. Incredibly, the daily average of visits for the last week is 80% higher than last month's daily average. My traffic has doubled since August of 2004.

Internet Explorer continues to lose ground to other browsers. This month I'm showing 65.7% for IE, 14.5% for Firefox, and 11.9% Unknown. Mozilla got 3.1%, and everything else was down in the noise under 2%.

There were 2,000 different search phrases used to access my site this month.  Some examples:

• The funniest search term I've seen yet is "percentage of rat in mcdonalds burgers". I don't know if the person was really wondering if there was rat meat in the burgers or if he made a mistake typing "fat." In either case, I near busted a gut on that one.
• Back in October I said I didn't know what "ham so won nude" was. It turns out that Ham So-won is a Chinese actress and singer who created quite a stir last year with her two books of nude photos. Sorry, no links. You'll have to track down those pictures yourself.
• The idea that somebody was searching for "erotic breast feeding pictures" is somewhat disturbing.
• "gas powered shoes"?
• The only way I know of to "stop loud attic turbine" is to replace it. They're only $20 or so down at Home Depot, and even an amateur home hacker like me can replace one in under 15 minutes.
• For the person who was wondering "how to manage a prima donna", my advice is "fire his ass."
• "rabbits in ketchup"?
• For the person wondering "how long does it take for a toenail to regrow," I can say from experience that it takes between six months and a year.
• "bad smelly gas caused by not smoking"?

I'm mildly disappointed that my Web traffic reports only list 1,000 of the search terms. I'm sure to be missing a few laughs.

I just spent a maddening few hours trying to get search engine friendly (SEF) URLs working for the Mambo installation on my test server.  SEF tells Mambo to generate more "natural" looking URLs for the pages.  That is, instead of something like http://charlie/index.php?option=com_content&task=blogsection&id=4&Itemid=49, the URL becomes http://charlie/content/blogsection/4/49/.  This is very important if you want search engines to index your site, as many search engines will discard a URL that looks as if it's pointing to dynamic content.  This shouldn't be terribly difficult to change, right?  Just enable Mambo's SEF option and set the proper options in the .htaccess file to enable Apache's rewrite engine.

The first problem I ran into was enabling .htaccess files on my system.  The documentation says to locate the <Directory /> section in the /etc/apache2/httpd.conf file, and add a line that says "AllowOverride All".  That's easy enough even for a Linux novice like me.  I restarted Apache and ... it didn't work.  The Web server continued to ignore my .htaccess directories.  Honestly, I wasn't terribly surprised by this, as just about everything I try to do under Linux involves reading the documentation to see how somebody thought it was supposed to work, and then scouring the Internet to determine what really works.

After looking at more documents than I care to remember, I came to the conclusion that Apache really is supposed to work that way.  So, figuring that something in my configuration was overriding my AllowOverride setting, I went searching.  I found the culprit in the file /etc/apache2/default-server.conf, which httpd.conf includes after the <Directory /> section.  The directives in default-server.conf "apply to all virtual hosts, unless deleted or overridden somewhere else."  I had to change the "AllowOverride None" directive in default-server.conf in order to make my .htaccess files work.  I don't know if this arrangement of configuration files is part of the standard Apache 2.0 distribution, or if it's specific to SuSE 9.2.  Either way, it's pretty darned confusing.

I got SEF working on the test server, but am having trouble getting it to work on the production hosting server.  My .htaccess file appears to be working, but apparently Sectorlink has disabled the Apache option that allows URL rewriting.  That's understandable, considering the security implications of URL rewriting if you configure it incorrectly.  If I can't get Sectorlink to enable the option, I'll have to come up with another way to process search engine friendly URLs.

I spent a lot of time working with the Mambo content management system on my Linux test server here, and yesterday installed it on a subdomain on my production hosting server.  The installation was not as simple as Mambo's install documentation would have you believe, requiring that I set a whole bunch of directory permissions and fiddle with a configuration file.  I wrote up a little article about my experiences, and will be posting it to the site.  I have things up and limping, but but I'm not pointing anybody to it until I iron out some of the more ovbious weirdnesses, two in particular:

• Designing a Web page template for Mambo is not trivial.  At least, it's not trivial for somebody with my meager HTML and CSS experience.  CSS, especially, is wonky enough to give a programmer fits.  I haven't yet figured out all the relationships and rules that determine what style will be applied where, and how defaults are supposed to work.  If you know of a good advanced CSS tutorial that goes beyond the simple stuff that's all over the Web, I'd sure like to hear about it.
• The instructions for enabling search engine friendly (SEF) URLs in Mambo are unclear and incomplete, and might be just flat out wrong.  I haven't been able to get SEF working on my test server or on the production server.  SEF is important if you want search engines to index your Mambo site, as most of them will discard a URL that looks like it's pointing to dynamic content.

I'm still not 100% certain that I'm going to convert this site to Mambo.  I will go to an online CMS at some point, but I want to be more comfortable with whatever system I choose before I do the conversion, as it's going to be a lot of work and I don't want to go through it all more than once.  I'll keep you posted.

I completed my article about asynchronous I/O in .NET, covering the oddities I mentioned in my January 13 entry.  The article contains an overview of asynchronous operations, describes how it's supposed to work in .NET, shows that it sometimes doesn't, and provides a workaround.  Take a look at Making Asynchronous File I/O Work in .NET: A Survival Guide over on DevSource.

Slashdot reported today that Take Two Interactive has acquired the rights to the Civilization franchise and that they will be developing Civilization IV. According to their press release:

The fourth game in the PC strategy series that has sold over five million copies, Sid Meier's Civilization IV is a bold step forward for the franchise, with spectacular new 3D graphics and all-new single and multiplayer content. Civilization IV will also set a new standard for user-modification, allowing gamers to create their own add-ons using the standard Python and XML scripting languages.

I'm disappointed.

Civilization II was a great game, not because the graphics were so good or because it could be modified, but because of its depth. It was (and arguably still is) the standard by which all other turn-based resource management games are judged. The graphics are primitive by today's standards, true, but that doesn't detract much from the game play. Unlike first person shooters, puzzle games, and even real time strategy games to some extent, the graphics in a resource management game are incidental to the game play, not part of it. They just need to be clear, accurate, and reasonably attractive. Graphics are not gameplay! This insistence on turning a game like Civilization into the next Doom 3 in order to appeal to the FPS crowd is folly. I want gameplay, not eye candy!

I reviewed the game Alpha Centuari here several years ago, and a few months later mentioned that I had decided not to try Civilization III because that game appeared to be yet another re-hash of Civilization II with fancier graphics. Subsequent reviews by other gamers bear that out: the game is pretty, but the game play hasn't changed. What a huge waste of developer resources.

The last thing the Civilization franchise needs is another pretty game that has no substance. I play simulations because I'm interested in the simulation. I'm all for eye candy, but not at the expense of game play. Those two programmers who are slated to work on the 3D graphics engine should be re-tasked to work on the game AI. With reasonably bright AI helpers that get more intelligent as the civilization advances, players are freed to worry about higher level things. As I said before: I want to manage an empire, not single-handedly construct every building and direct every battle.

With better AI assistants (like a city manager that can actually do a credible job of managing a city, and automated units that can be given specific tasks like interconnecting cities with roads rather than mindlessly "optimizing" random pieces of real estate), the game could provide a broader simulation, including advanced trade, more opportunities to interact positively with other civilizations, more domestic concerns, and much less focus on war. But if the game is going to focus on war, then at least give me the ability to direct a campaign and leave the individual battles to AI helpers. The game could be so much better if resources were spent on gameplay rather than on flashy graphics and a scripting engine.

What makes a turn based strategy game immersive is the depth of the simulation, not the attractiveness of its graphics. Add more elements to the simulation, create graphics that allow me to monitor those elements, and concentrate on balancing the gameplay. That is the recipie for a successful resource management game. Leave the cutting edge 3D rendering to the games that can actually benefit from it.

If you're writing a Web diary (blog), you're probably interested in how many people visit your site.  I've been writing my Random Notes for over four years, but had never tracked visitors on a regular basis.  I did have something that'd tell me how many hits I got, but hits don't tell the real story.  Every time the Googlebot reads a page, it counts as a hit.  The important numbers are unique visitors and visits.  As part of my hosting package at Sectorlink, I get very detailed reports on the number of visitors, visits, pages served, and hits.  The reports differentiate between hits by automated crawlers and visits by client programs like Web browsers.  This gives me a very good idea of how many people are visiting the site.

I started keeping close watch on the number of visitors when I added my RSS feed last summer.  At that time, I also started submitting my site to search engines and aggregators such as Weblogs.com, syndic8, and Technorati.  Since then, I've doubled the number of visitors to my site.

A colleague recently sent me a link to a feed submitter that will submit your RSS feed to 15 RSS aggregators as well as Google search and Yahoo search.  You simply enter your feed URL and your email address, and the program automatically submits it for you.  I did that last week, and two days later I noticed an increase in activity to my site.  I can't guarantee that submitting was the cause, but it sure looks suspicious.

A recent news story said that on average, American adults are 25 lbs lbs heavier now than the adults of 40 years ago.  That's a pretty sensationalist headline, evoking images of a nation in which every adult is 25 lbs overweight.  The media, as I've pointed out before, have a poor understanding of the term "average" in general, and they seem incapable of presenting a balanced story underneath their sensationalist headlines.  For example, some of that average weight increase is undoubtedly due to a reduction in malnutrition and also to an increase in average height.  If we all were an average of a foot taller, that 25 lb increase wouldn't be an issue.  The real issue is the percentage of people who are considered overweight or obese today compared to 1960.  But it's difficult or impossible for many reasons to make an accurate comparison of today's population with the population in 1960, so any comparisons you see are likely pulled out of thin air or somewhere that the sun doesn't shine.

More often, comparisons are made against 1990 for some reason.  The number tossed about these days varies depending on the source, but somewhere between 40 and 70 percent of American adults today are considered "overweight" or "obese".  The number who are considered "obese" (more than 20 lbs overweight) is reportedly between 10 and 30 percent.  That's roughly double the number from 1990 if you believe the reports, although again there are problems comparing today's population with 1990's population, the two biggest being changing criteria and detection bias.

Although today we have the Body Mass Index (BMI - more below), there was no generally recognized standard in 1990 for determining who was overweight or obese.  Different doctors used different criteria.  One could argue that the overs and shorts would balance--that is, that on average the doctors who had more stringent criteria were balanced by those who were more lenient.  It's a nice theory, but probably impossible to prove.  In any case, the lack of a generally accepted standard makes any comparison against 1990 somewhat suspect. 

The other problem with comparing today's population to 1990's population is detection bias:  we're seeing more fat people these days because we're looking for them.  If you go looking for something, you're almost guaranteed to find more of it than if you weren't looking at all.  Scientific studies in general, and medical studies in particular, often suffer from detection bias.  Another form of detection bias is trying to compare a population that you can see (today's Americans) with a population that you can't see (Americans from 1990).  It's possible to sample particular populations from historical records (military physicals, hospital admissions, etc.), but then you have to prove somehow that the sample is representative of the entire population.  That, too, is a difficult proposition.

The most widely accepted standard used today to determine one's "relative fatness" is the Body Mass Index, or BMI.  The BMI procedure calculates a height-to-weight ratio, and then uses that calculated number to put you into one of four categories:  "Underweight," "Normal weight," "Overweight," or "Obese."  Calculating the height-to-weight ratio is a useful measure of relative fatness, although it's not a definitive guide.  A body builder, for example, would be inordinately heavy for his height.  Still, the height-to-weight ratio is a good starting point.  In general, a higher BMI value is an indication of excess body fat.

My only real issue with the BMI standard is the grading scale, which I think is skewed towards an unrealistically thin ideal.  For example, my weight of 185 lbs and height of 5' 9" results in a BMI value of 27.5--halfway between overweight and obese.  Now I'll be the first to admit that I could stand to lose a pound or three, but I'd have to get down to 169 lbs to get out of the "Overweight" cagegory.  At 169 lbs, I'd be a lean, mean, biking machine with the physique of a professional bicycle rider.  That might have been realistic when I was 30 or 35, but certainly not today.  The BMI scale needs to be adjusted up a few points and probably adjusted for age as well.

That's my long winded way of saying that I don't put too much stock in the reported percentages of overweight or obese people, or in comparisons between today and 15 or 40 years ago.  It would be nice to know what those number really are, but I don't think that we can obtain them to any degree of certainty.  That said, it looks to me as though there are many more overweight and obese people today than there were 15 years ago, and I'm wondering why that is.  I'll be exploring that over the next few weeks.

Definitions are slippery things. The word "habit," for example, is defined as "an acquired mode of behavior that has become nearly or completely involuntary." A "compulsion," on the other hand, is "an irresistible impulse to perform an irrational act." I wonder what you'd call an irresistible impulse to perform a rational act, but I digress.

The first thing I do when I get up in the morning is ... ummm ... well, the second thing I do every morning is brush my teeth. Almost without fail. This is behavior born of long practice. Some people can get up, go for their morning walk, and have breakfast before brushing their teeth. I can't. Really. If I don't brush my teeth right when I get up, it throws my whole rhythm off. I've learned not to fight it. I've often wondered if my need to brush first thing every morning is bordering on compulsive behavior. Certainly there's no logical reason why I couldn't wait until after my morning bike ride.

That's not the only behavior I sometimes wonder about. Before I turn the ignition key off in the car, I always turn off the lights, radio, windshield wipers, back window defroster, air conditioner and anything else that's "on." I even turn the fan control off and move the temperature dial to its lowest setting ("off" if you view it as the heater control).    I developed this habit during years of flying in private planes with my dad, and then as a pilot myself. The checklist says to turn off all electrical devices, turn off the ignition, and then turn off the master switch. This is absolutely essential in most airplanes, because gyros and other things will draw current if the master switch is on. Many a pilot has had to have the FBO come jumpstart his airplane because he left the master switch on. But it's more than a habit for me now.  It's an irrational ritual in a modern automobile where everything's keyed to the ignition switch, especially when you consider that the first thing I do when I start the car is turn on the radio, air conditioner, lights, and windshield wipers.

I haven't had a locker and closet inspection since I left the Air Force Academy in 1982, yet I still hang my pants (you have no idea how long it took me to stop saying "trousers") with the zipper facing out and the legs hanging to the left. When Debra and I first got married, she caught me re-hanging some pants that she'd hung wrong. Let's just say that I don't let her catch me doing that anymore.  Habit or compulsion?

So am I twisted? Do these and other behaviors indicate a mild form of obsessive compulsive disorder? I've come to accept that they probably do. I've also come to believe, that everybody exhibits signs of one or more "disorders," and that complete normalcy is a myth. A person who had no idiosyncracies would be incredibly dull. The most creative, dynamic, driven, or productive people I know all are slightly unhinged in one way or another. Society wouldn't change otherwise. Striving for "better" or "different" when what you have is "good enough" isn't entirely rational.

The line between idiosyncratic behavior and mental illness is pretty fuzzy. I accept and even embrace my little weirdnesses, but I also keep them in check. I'm not sure my friends would understand if I rearranged their closets the next time I came for a visit.

Although I'm impressed with Mambo so far, and think that it will do what I need, I'm being somewhat cautious with my implementation.  I've decided that I'm going to install it on a production Web site, but that first installation won't be here.  I want to make sure that I have a good understanding of its strengths and weaknesses before I go through the trouble of reworking this site and converting all of my content.  Few things are more painful than having to revert after a failed migration.

If you're looking for a content management system for your site, be it a personal site like this one, a large corporate site, or something in between, there's probably a solution out there for you.  I found the following resources very useful while I was doing my research.

• opensourcecms.com (built with Mambo) provides "try before you install" demos of many open source CMS packages including portals, blogs, forums, Wiki, and others. This site is specific to systems written in PHP, using MySQL, and running on Linux. It contains a short description of each product, and a fully operational demo that lets you drive the product around and even fool with the administrative pages. Their demo server re-installs the demos every two hours, so any changes you make to the configurations aren't permanent. You don't have to be afraid of goofing something up.
• CMS Matrix has a list of literally hundreds of content management tools, and a tool that lets you search and select by feature set, or select different packages to compare features.  It can help you narrow the field quickly so that you can concentrate on examining just a few products in depth.
• Open Source Content Management is "the international association for open source content management." They have lists of projects, a news feed, a features matrix, calendar of events, standards, and other things you might expect from an association of CMS developers and users. They also apparently have a Wiki, although I've not been able to access it.
• cmsinfo.org is an Internet community of CMS developers and users that provides news and information about open source web log programs and content management systems. It had some good news coverage in the past, but it looks like updates are spotty. I don't see any news since December of 2004. Still, they have a list of open source CMS systems and some old news articles that will give you some idea of what's available.
• CMS Watch is a commercial endeavor that supplies consulting and sells some industry-specific reports online. Their site has industry news, product lists, some general information about CMS topics, and articles about and reviews of CMS products.

Beyond that, you're on your own. I'm still learning the lingo, and I don't have much experience with any of the products yet except CityDesk and my ongoing education (for work)in Microsoft Content Management Server.

If you're wondering what I'm talking about, check out the Wikipedia article Content management systems.

I spent an incredibly frustrating few hours on Wednesday night, and another couple of hours here tonight trying to get Mambo installed on my SuSE Linux system.  This turned out to be way more difficult than I had imagined, not because Mambo itself is difficult to install, but because getting Apache, PHP, and MySQL all talking together is something of a black art.  At least, it's not well documented in any of the normal places I looked (like on the PHP, Apache, and MySQL Web sites).  They all have bits and pieces of information that, in retrospect, combine to make it all coherent.  I had just about given up on it when I stumbled across a beautifully written document on Novell's Web site:  Installing Apache, PHP, and MySQL on SUSE LINUX Professional.

Kevin Millecam's instructions led me step-by-step through installing the relevant packages from the SuSE 9.1 distribution CD, including configuring the servers and making sure all the right permissions were set.  Even with the mess my previous experiments had made of the configuration files, I was able to get it all working in just a few minutes.  If you're looking to install PHP and MySQL on your Linux box, read this document.  It's specific to SuSE Linux, but a lot of it will be useful if you're installing on some other distribution.  Highly recommended.

Kudos to Novell for sponsoring the Cool Solutions community site.  We need more encouragement of authors to submit this kind of helpful documentation, and major Linux vendors would do well to make it available.

It seems like bug tracking is always a difficult problem.  In the years that I've been with Catapult, we've used all kinds of different bug tracking methods, from Excel spreadsheets and Access databases, to full-blown systems that cost hundreds or thousands of dollars per seat.  One of the problems is that we often are engaged at a client's site and have to use whatever system they have installed.  But even internally, for many different reasons, we haven't settled on a single bug tracking solution.

Part of my assignment with a new client is to evaluate, suggest, obtain, and install a bug tracking system for the project.  After looking around a bit, I've narrowed my focus to three products:  Sharepoint, FogBugz, and Bugzilla.  It's not an easy choice.

The advantage of using Sharepoint is that the client already has that software installed.  My last client also had Sharepoint, and they had developed (or purchased, I'm not quite sure) a simple but effective bug tracking system.  Since the client, and we at Catapult, are using Sharepoint for collaboration, adding a bug tracking web part seems like a no brainer.  But I haven't determined yet whether the system we used at the last client has all of the features we'd need to make it a company-wide solution.

I've heard good things about FogBugz, and the few consultants at Catapult have used it at clients' sites have reported that it works well.  At $99 per named user (drops to $80 each for 100 users), it's a heck of a bargain.  It's Web based, will run on the Windows servers that are prevalent at Catapult, and it appears to be robust enough to handle the many different projects that we have.  $8,000 isn't a lot of money to pay for a piece of mission-critical software that will be used by 100 consultants.  At going rates, that's between 80 and 120 hours of work.  A good bug tracking system can save you that on the first project.

And then there's Bugzilla:  free, open source, and quite good as I understand.  It's designed to work on Linux, although it supposedly can run under Windows.  I spent a few hours trying to get it running on my Windows 2003 Server box, with the final result that it's up and limping.  It requires that you install Perl and MySql, which aren't too much trouble on a Linux system, but are a little more difficult under Windows.  (ActiveState is the place to get Perl for Windows, by the way.)  The Bugzilla documentation strongly recommends running it on Linux, and there's some resistance at Catapult to installing a Linux server in a mission critical role, mostly because our entire infrastructure is built on Microsoft server technologies and our administrators aren't as familiar with Linux as they are with Windows.  Even if we got Bugzilla running on a Windows server, we'd still have to maintain the MySQL database and contend with Perl--something with which Windows system administrators aren't intimately familiar.

If I had to make the decision today I'd go with FogBugz because people I trust have recommended it and it looks like it'd cause the smallest number of headaches down the road.  But I'm not the one writing the check.  I'm pretty sure that Bugzilla is out of the running, based on our Windows focus, but I'll complete my evaluation.  It'll be hard to beat the price of the Sharepoint solution, though, especially if it has the features that we need.  Next week should prove interesting.

Several of my friends and business contacts have asked me to update my contact information on Plaxo.  With the trouble that Debra and I have keeping our address books up to date and synchronized, I'm seriously considering taking a closer look at using Plaxo myself.  It's a very attractive idea:  keep all of our contact information in one place where we can get to it from any computer in the world.  That sounds like a whole lot better solution than what we currently have:  contact information spread among three different computers, two mobile phones, and a couple printed copies of old address books complete with cross-outs, additions, and indecipherable chicken scratches.  Add to that the inevitable confusion that results from conflicts due to one of us updating somebody's phone number or email address and not telling the other, and you begin to see why we spend so much time trying to figure out how to get in touch with people.

According to the Web site, Plaxo "plugs seamlessly into Outlook or Outlook Express."  That's nice, I guess, but I don't use either of those programs here at home.  I use Outlook at work because I have to, but for my personal mail I'm quite happy with Mozilla Thunderbird.  Even if it can't integrate with Thunderbird, though, Plaxo might be the way to go just to have everything in one place.  It'd be easy enough to add the few people I correspond with on a regular basis to my Thunderbird address book.

As attractive as Plaxo looks at first glance, I'm still a bit nervous about storing all my contact information on somebody else's server.  Not because I'm afraid they'll lose the information.  No, I trust their backup procedures, and I can always replicate the information on my local machine.  What worries me is the potential of that information being misused by an unscrupulous employee, a hacker, or the Federal government if they come in with a search warrant.  I wonder just how hard it would be for the FBI to seize somebody's address book.  You can learn a lot about a person just by seeing who he's talking to.

Back in July, I started using CityDesk to create this Web site. Since then, I've converted all of my Random Notes entries to CityDesk . I've also either converted all of the other pages, or copied them directly into CityDesk as HTML files. I now manage the entire site in CityDesk, and I've deleted the old FrontPage site from my hard drive. (After backing it up, of course.)

After working with CityDesk for a little over six months, I give it mixed reviews. My impressions of the program itself are mostly favorable. It's easy to install, simple to use, and I'm able to create a new site very quickly. Template editing--especially that wonky CityScript language--is quirky, but easy enough despite its oddities. The program's user interface is okay, but certainly not up to the standards that Joel Spolsky (the owner of Fog Creek Software, and principle designer of CityDesk) espouses on his Web site. There are missing keyboard shortcuts, context menus lacking relevant options, features that are accessible only by the mouse, and minor bugs in the editor that drive me batty. Still, the interface is easy to use, mostly intuitive, the program works most of the time, and I've never lost any data.

I especially like that the database format is fully documented, making it possible for me to write programs that can modify articles or create new articles from information that's in the database. In particular, I'm working on a topic index, table of contents, and image index generator. I might not complete those projects, though, because . . .

My only real gripe with CityDesk isn't with the program at all, but rather with the publishing paradigm. CityDesk is an offline content management system. I create articles, store them in a local database, and then press a button to have CityDesk generate static HTML pages it then uploads to my Web site. This model has certain advantages, among them simplicity and an off-server backup of the content. It also serves the lowest common denominator: anybody who has FTP access to his Web server can publish content using CityDesk. CityDesk does these things very well--certainly better than any other reasonably priced Web site creation tool that I've seen.

There are some problems with the static model, though, and some of those problems increase as you add more content. The one I'm currently fighting is changing the style sheet or article template. If I change either of those files, CityDesk will re-generate every page that uses that template, and then upload every page to the Web server. That's the nature of static content. With well over 1,000 Random Notes entries, you might imagine that I'm a little reluctant to make any changes to the style sheet or template.  That's disappointing, because it would be nice to make seasonal or topical changes to the templates.

A related problem is the amount of file space all those pages take. Every Random Notes entry includes the HTML code to render the page heading and side panel on the page. That code takes approximately two kilobytes per page. Not a problem if you have just a few pages, but with over 1,000 pages on the site, I'm talking over two megabytes of wasted space. With 500 megabytes of disk space for my domain, I'm not too worried about a couple of megabytes (not when I post single pictures that take more than that), but it is a concern if I decide to create a template with more stuff on it. Every byte used in the template is multiplied by the number of articles posted.

Another problem of size is the time and local disk space it takes for CityDesk to generate the pages and determine what needs to be uploaded.  It currently takes almost two minutes for CityDesk to generate pages when I click on the Publish button.  CityDesk apparently re-generates every page into a temporary directory so that it can quickly upload any required pages once it connects to the server.  That's a worthwhile optimization, but the wait time will continue to increase as I add more content.

The static model also restricts what I can do with the site. Specifically, I can't post entries if I'm away from home, and I can't include comments, "track back," or other dynamic features that other Web sites and dedicated blog sites provide. A laptop would allow me to post entries when I was away, provided that I had it with me and access to an Internet connection. That presents some backup issues, but those are minor. I don't relish the idea of lugging a laptop around all the time, though. I like traveling light.

So I'm reconsidering my options and am again evaluating content management solutions for the Web site. The frontrunner at the moment is Mambo, which on the surface looks a lot like the PostNuke package that I evaluated last year, but appears to be more secure and easier to use. Whatever I decide to do, I'll either install it on my test server here at home first and learn to use it, or I'll put it on one of my other domains and drive it around a bit before trying to convert this Web site. Stay tuned.

On January 3, I received an email message from Mark Nelson, who had decided to auction his DataCompression.info link farm.  He'd been maintaining links and news about data compression for several years, but hadn't been able to keep it updated recently.  The auction included the datacompression.info domain name, his database of data compression related information, the software to convert his database information to Web pages, and archived news items.  He decided to auction it all on eBay because he couldn't come up with a decent plan to decide who would make the best caretaker.  As he said in his message:  "I don't think this domain name is particularly valuable, but it does currently enjoy Google's top PageRank for a few data compression related terms."

I had considered bidding on the domain when it was around $200.00, figuring I could probably make some money selling Google ads on the site.  I decided against it for two reasons:  I have enough things to keep me busy without searching around for data compression related information all the time; and I didn't think enough people would click through on the Google ads to pay for the hosting and bandwidth.

I guess the domain name was worth more to somebody than either of us thought.  The auction ended on January 9 at a price of $5,800.00.  The site is now sponsored by a video converencing company called Visicron.  It'd be interesting to discover who's behind the $5,800 purchase price and why he thought the domain name was so valuable.  Whatever the case, I hope he maintains and continues to update the site, as it's a valuable resource.

Ah, the joys of home ownership.  A couple of months ago a small animal apparently got trapped in the garage and scratched the side entry door in an attempt to get out.  We never did determine what kind of animal was in there, but it sure did a number on the door.  In addition to the side entry door, the roll-up door on the back bay had rotted to the point where it was coming apart.  Replacing those two doors was the only required project on my list during my two-plus week vacation.

I've installed exterior doors before.  They're a little more difficult than interior doors, but usually not too much trouble if I take my time and pay attention.  Provided, of course, that the walls are plumb and the rough opening is reasonably square.  I had to install the door twice because I got in a hurry.  I put the thing in, checked that it operated without binding, attached the trim, installed the doorknob, and then noticed that the door wouldn't latch.  I made the mistake of checking that the door frame was level, with the assumption that the slab on which it rests is level.  I know better than that, having installed doors around here before.

I repaired my door installation last Saturday (January 8), although I didn't complete it (see below), and started on the garage door.  I was surprised that I could buy an entire garage door kit, including door, rails, and springs, for under $200 at Lowe's.  Not having ever installed one before, I was concerned that I would need help.  But, being the adventurous type, I opened the package and started reading the instructions.

Installing a garage door isn't really that hard, it just takes a long time.  It took me about an hour to remove the old door, and then about 10 hours total on Saturday and Sunday afternoons to put the door panels together, place them in the opening, attach the rails and springs, and ensure that the door operates smoothly.  One thing that I was very concerned about was working with the springs, as I'd heard that people had been injured or killed working with garage door springs.  As it turns out, there are two types of garage door springs:  a torsion spring and a traditional spring.  The torsion springs are very dangerous, and you should not attempt to install or adjust them yourself.  But the traditional springs are no problem as there is no tension on them while you're installing the door.  As long as you follow the safety procedures in the instructions and install the safety cable, it'd be very difficult to injur yourself with a standard garage door spring.

Installing the standard entry door involved another twist:  the wall was framed such that there was no space for the threshhold.  As a result, it hung over the edge of the slab about three inches.  I had planned to attach some wood to the slab underneath the threshhold to provide support.  I'd seen that done before.  But my concrete contractor neighbor suggested that I pour a small concrete step there.  With his help this afternoon, I built a wood form for the step, drilled some holes in the slab to attach rebar so that the step would be attached to the slab, and mixed two 160-lb bags of concrete for the step.  Given the cold temperatures, I'll probably wait two days before I pull the form off.  After that, I just have to paint the new trim and I'll be done.

The second phase of house remodeling starts next week.  I wonder if this will ever end.

I managed to get into some poison ivy last Sunday while I was out mountain biking at Walnut Creek Park.  I've gotten into the stuff before, although I've never recognized it at the time.  Usually I come home and shower right after mountain biking.  If I'd gotten into the poison ivy, I'd notice a little rash or a few blisters the next day.  This time, though, I came home, changed clothes, and went out to work in the garage.

Giving those oils time to work is a bad idea.  By Wednesday I had very nasty patches on my right arm and right leg, and they seemed to be spreading.  I finally broke down and went to the doctor, who gave me a cortisone shot.  It's probably a good thing that I'm not a professional bicycle racer, or I'd probably be in violation of their doping rules.

By the way, anybody who tells you that poison ivy can't spread by contact would have a tough time convincing me.  I originally got it on my right forearm.  A couple of days later I managed to get it on my right thigh.  I'll leave you to imagine the common posture that allowed that particular contact.  In any case, it's been almost a week now, and this rash still itches like mad.  Stay away from the poison ivy.  If you do get into it, be sure to wash thoroughly as soon as possible afterwards.  You don't want to go through what I've been through the last week.

I was doing some speed testing today, trying to saturate a USB port by writing to multiple flash RAM drives at the same time.  I took the opportunity to explore the asynchronous I/O support in the .NET Framework, and found out that it doesn't quite work as advertised.  I could do asynchronous reads, but asynchronous writes to the USB drives would block.  Indulging my curiousity, I decided to see if it worked on network drives and the local hard drive.  I got the same results:  asynchronous reads work as expected, but asynchronous writes block.  At least, the writes block on network I/O.  It's kind of difficult to determine if the writes block on the local drive, as the thing will write 100 megabytes almost instantly.  Windows has a serious problem with writing larger blocks (it appears to depend on how much memory you have), but that's a topic for another day.

A few minutes with Google revealed the Microsoft Knowledge Base article Asynchronous Disk I/O Appears as Synchronous on Windows NT, Windows 2000, and Windows XP.  The article explains the many reasons why asynchronous I/O might appear synchronous, and gives hints for resolving the problems.  One of the primary reasons is caching.  If the file is opened using caching, then for complicated reasons the cache can work against you and the I/O operation will proceed synchronously.  This is an interesting case of an operating system optimization (caching) interfering with an application-level optimization (asynchronous I/O).  The article explains how to get around the problem by turning off caching for the file, but doing so imposes some restrictions on the I/O buffer and the size of reads and writes.  It doesn't appear that a .NET program can meet those requirements without resorting to the Windows API to open files and allocate memory.

Fortunately, there is a workaround in .NET--spawn a thread and have it do the I/O synchronously.  By creating multiple threads, I was able to get back to my testing of the USB drives.  But that project is going to take a back burner while I explore this asynchronous I/O oddity.  The reason?  Nobody's paying me to swamp my USB drive, but I can probably sell an article about how to do asynchronous I/O in .NET programs.

I picked up another 256 MB Memorex TravelDrive at Fry's the other day. I selected the Memorex unit for the simple reason that it's the same as the first one I bought. Might as well standardize on something, right? The Memorex Thumb Drive, though, is a poor choice.

Why? Because the fat body on the things makes it impossible to plug two of them side-by-side into a USB hub. You won't get more than two of these babies into a four-port hub if the ports are arranged side-by-side. No problem if they're arranged one on top of the other. Something to keep in mind the next time I go looking for portable storage.

Hey, Memorex. You goofed on this one.

The middle part of the 9/11 Comission Report (Chapters 2 through 8) are a rambling discussion of how Usama Bin Laden and his al Qaeda group formed, what made them able to attract a following, their early attacks, our responses, and finally a detailed (although still rambling) account of how the 9/11 attacks were planned and carried out.  There are few surprises here.  The most important thing I got out of those chapters was an appreciation for how well planned and organized the operation was.

When information started leaking out of the 9/11 Commission investigations, some people made a lot of noise about missed opportunities to capture or kill Bin Laden and some of his chiefs during President Clinton's second term.  The Report details some of those opportunities, and explains very well why they weren't taken.  The reasons fall into three categories:  insufficient information on his location, insufficient resources to carry out the attacks, and negative political consequences.  In almost all cases, the political consequences were negative for the country as a whole, not for the President.  On the contrary, Clinton's approval rating domestically would have climbed quite a bit if we had managed to capture or kill Bin Laden.  But the international repercussions for the United States would have been staggering, although one wonders if they would have been any worse than what's happened in response to our Iraq venture.

I often wonder if Bin Laden miscalculated our response to the 9/11 attacks.  Given our responses to their previous attacks, he had every reason to believe that our response to 9/11 would be weak and ineffective.  When al Qaeda carried out the embassy bombings in Africa, we lobbed a few cruise missiles at a manufacturing plant and a couple of terrorist training camps.  We did nothing after the USS Cole bombing.  I suspect that he expected some response to 9/11, but I doubt he imagined that we'd send troops to invade Afghanistan looking for him.

The Report spends a lot of time detailing missed opportunities or "failures" on the intelligence side--information that different agencies had, but that we didn't put together prior to the attacks.  The report is quick to point out situations in which, had we been able to put the information together, we might have or could have prevented the attacks.  The members of the Commission recognize (in writing, at least) that these things that are obvious in hindsight weren't so obvious at the time.  I question whether they actually believe that.

There's no doubt that had the right people in the right agencies been able to share information, we would have learned of the attack plans and prevented them.  However, the relevant information was in bits and pieces spread out over many different agencies, many of whom were prevented by law from sharing information.  Those restrictions on information sharing evolved over the last 20 or 30 years to prevent or lessen abuse and government intrusion into our private lives.  The question isn't whether or not we could have identified the threat and prevented the attacks, but whether we want a national intelligence structure that has the capability to tie together the few bits of information that we did have.  Granted, it would have been nice to prevent the attacks, but at what cost to our own privacy?  The intelligence structure that could have detected and prevented the attacks is a frightening thing in a free society.  More on that when I talk about the Commission's recommendations.

Two questions more important than why we weren't able to stop the 9/11 attacks after the flights were hijacked are how the 9/11 hijackers managed to get into the United States undetected and get through airport screening while carrying dangerous weapons.  Note that there is no evidence that any of the hijackers had firearms.  According to eyewitness accounts (phone calls from the planes), the attackers were armed with box cutters.

The more difficult part of the operation was getting the operatives into the United States undetected.  The people who planned the operation weren't stupid.  They knew that if they tried to use operatives who had participated in other operations, the CIA or some other intelligence agency probably would know of them.  So they used fresh recruits.  Before 9/11, it was absurdly easy to get a student visa, especially if you had a little bit of money and an understanding of the system.  Although some of the hijackers had overstayed their visas, that wasn't a big risk--the INS wasn't terribly interested in running down people who overstayed their visas.

The attackers entered the United States legally, after a fashion.  They got their visas under false pretenses, which technically is illegal, .  but they did fill out all the proper forms and get all the right approvals.  Once in the country, they used several different means to set up housekeeping, get drivers' licenses, open bank accounts, rent apartments, etc.  They kept their heads down, spent time together and in the gym, and waited patiently for their time.  Once they were in the country, they did nothing to draw attention to themselves.

It's well known that there were supposed to be 20 hijackers.  One was denied entry into the U.S. after arriving from overseas.  There is some evidence that the original plan was to have at least five more, possibly including another pilot so that there would have been five rather than just four hijackings.  It appears that some of the planned hijackers were denied visas.  A suspected fifth pilot, Zacarias Moussaoui, had attracted the attention of the FBI in August 2001 due to his unexplainable interest in learning to fly a Boeing 747.  It's unlikely that by August Moussaoui was slated to be part of the 9/11 attacks, although he probably had that intention when he came to the U.S.  He was arrested for overstaying his visa, and a deportation order was signed on August 17.  The FBI suspected Moussaoui of harboring ill intent and wanted to pursue a more detailed investigation, but they didn't have enough evidence to give them probable cause for a search warrant.  After 9/11, information received in response to inquiries made before 9/11 gave the FBI probable cause, at which time they confiscated and examined his laptop computer which apparently contained some information about the plot.  Had the FBI obtained that information before 9/11, it's likely that they would have derailed the attacks.

Getting through airport security apparently presented no problems.  Over half of the hijackers were actually flagged for additional scrutiny, either by a computerized prescreening system known as CAPPS, or by the security screeners.  The only consequence of being flagged by CAPPS was that the individual's luggage was not loaded onto the airplane until it was confirmed that the passenger had boarded.  This was a federally mandated procedure that was intended to prevent a Pan Am 103 type incident where the terrorists checked luggage containing a bomb but didn't board the flight.

The security checkpoints at Boston and Newark lack video surveillance, so it's impossible to say whether any of the hijackers on those flights received any special attention.  Of the five that passed through the security checkpoint at Dulles, three set off the initial metal detector and were directed to a second detector.  One cleared the second metal detector.  The other two set it off, were hand-wanded, and allowed to proceed.  After 9/11, the Commission asked a screening expert to review the videotape of the hand-wanding.  He found the quality of the screener's work to have been "marginal at best."

Understand, this was before 9/11, when it was perfectly acceptable to carry screwdrivers, nail clippers, and even small pocket knives on board.  Back then, nobody seriously considered the possibility of hijacking an aircraft with a few knives.

Again, we see that the hijackers studied our procedures, learned our weak points, and exploited them.  It's important to understand that this was a well planned and well executed attack.  All of the evidence points to the 9/11 attacks having been years in the planning.  I've often heard people express the opinion that the 9/11 attacks were carried out by a bunch of poorly educated third world terrorists who just got lucky.  The evidence contradicts that opinion.  The people who planned and carried out these attacks were intelligent, methodical, and very patient.  It's stupid and dangerous to think otherwise.

The first chapter of the 9/11 Commission Report gives a very detailed account of what we know occurred before the hijackers boarded the airplanes, the events that unfolded on the airplanes, and the responses of the different corporate and government agencies that were involved.  Our understanding of when the individual flights were hijacked is based on several things:  unguarded transmissions from the airplanes by the hijackers (who apparently thought they were communicating with the passengers), transponder data or lack thereof due to the transponders being turned off, radar records showing course deviations, lack of response to instructions by FAA controllers, and communications from flight attendents and passengers to people on the ground.  In addition, we have varying degrees of detail about happenings aboard from those cabin-to-ground communications.  Finally, the cockpit voice recorder and flight data recorders that survived the crash of Flight 93 in Pennsylvania provide some additional insight.

A look at the timeline of events answers one of the biggest questions I've heard expressed about the events of the day:  Why didn't the military do anything about it?  The simple answer is that there wasn't enough time between the flights' being taken over and the crashes for controllers to determine that the flights were hijacked, and for that information to be communicated to people responsible for deciding what to do about it.  For example, American Airlines Flight 11 departed Boston at 7:59 A.M.  The last routine radio transmission occurred fifteen minutes later at 8:14, and takeover occurred within minutes after that.  At 8:19, a flight attendant notified American Airlines of the highjacking by making a call from one of the seat phones in the rear cabin.  Boston Center learned of the hijacking five minutes later.  At 8:38, Boston Center notified the military, which scrambled fighters at 8:46--less than a minute before Flight 11 crashed into the North Tower of the World Trade Center.

United Flight 175 took off from Boston at 8:14--the same time Flight 11 was being taken over.  Flight 175 was taken over sometime between 8:42 and 8:46 while controllers and others were trying to figure out what to do about Flight 11.  The first indication of anything wrong with Flight 175 was at 8:47, when the transponder code was changed.  A flight attendant notified United of the hijacking at 8:52.  It crashed into the South Tower at 9:03.

Thirty two minutes (the time between likely takeover and crash of Flight 11) is an astonishingly short amount of time for information to flow from controllers to decision makers to alert fighter crews who then have to scramble and find an aircraft.  Otis Air Force Base where the fighters were based, is 153 miles from New York City.  At the F-15's maximum published speed of 1,650 MPH, it would take almost six minutes to cover that distance.  Obviously they had no chance to stop Flight 11 which crashed just as the fighters were taking off.  It's possible, although highly unlikely, that they could have located and engaged Flight 175 before it crashed at 9:03.  In addition, it's unlikely that anybody would have given the order to shoot down a domestic passenger aircraft over a major metropolitan area, especially with less than 15 minutes in which to make that decision.

American Flight 77, which departed Washington Dulles at 8:20 is a somewhat different story.  It was taken over sometime between 8:51 and 8:54, when it made an unauthorized turn to the south.  The transponder was turned off at 8:56, making the aircraft very difficult to identify on radar.  Although American Airlines was aware at 9:05 that the flight was hijacked, there is no indication that the FAA knew about the hijacking.  The FAA did learn that the flight was "missing," and notified the military of it at 9:34.  A National Guard C-130 that had just taken off for Minnesota saw the aircraft, identified it as a Boeing 757, and began to follow.  Three minutes later, Flight 77 crashed into the Pentagon.  There were fighters in the air near Washington, but due to the mass confusion caused by the first two crashes, they were looking in the wrong place for the wrong thing.  And again, it's doubtful that they would have engaged the airplane over metropolitan Washington.

In my opinion, the only reason the hijackers didn't go four for four was that United Flight 93 was 25 minutes late departing.  It didn't leave the ground until 8:42.  The pilots actually received a warning about increasing cockpit security just minutes before the flight was taken over.  The passengers ultimately revolted and attempted to retake the airplane after learning of the first two hijackings and crashes.  Had the passengers not revolted, it's unlikely that the military could have located Flight 93 and received authorization to shoot it down in time to prevent it from striking its target (likely the Capitol or the White House).  There's no doubt in my mind that the hijackers would have been successful had the flight departed on time from Newark.

At the time of the hijackings, our nation's air defense was designed to repel a military attack from outside.  There hadn't been a domestic hijacking in over 10 years, and in any case a hijacked aircraft wasn't considered a dangerous weapon.  As the 9/11 Commission Report states:

NORAD and the FAA were unprepared for the type of attacks launched against the United States on September 11, 2001.  They struggled, under difficult circumstances, to improvise a homeland defense against an unprecedented challenge that had never before encountered and had never trained to meet.

This is a common theme throughout the Report:  al Qaeda studied our defenses, identified our weaknesses, and exploited them to the fullest advantage.  More later.

The cable guy was out disconnecting our cable TV service as I left the house to meet some friends for breakfast this morning.  Debra and I decided that it made no sense to continue paying for a service that we don't use.  Neither of us has watched television in over a year.  No, we don't miss it.  There are plenty of other more interesting and entertaining things to do with our time:  read books, go for walks with Charlie, ride bicycles, watch a rented DVD, or fiddle with one of our many hobbies (hers revolving mostly around African Violets, mine with computers and ham radio).

Friends have told me that I'm missing out on a lot of good educational programming on the History Channel, Discovery, and PBS shows such as Nova.  With the exception of Nova, which periodically has a good show, I've found the "educational" programming to be written for people who have very short attention spans.  Topics are covered superficially at best.  Whenever things start to get really interesting a commercial break comes up and the show switches topics upon return.  The Internet is a much better educational device than is the television.

With few exceptions, the only thing I find entertaining about broadcast television is that people actually watch it.  There have been a few shows over the years that were well written and I actually looked forward to watching.  Pay television (HBO, Cinemax, etc.) were fun for a while, but I found myself spending entirely too much time watching those "free" movies.  After a few months I'd seen most of the regular movies, and the few new movies each month weren't worth the price of admission--scheduling my time around the movie schedule or putting in a tape to record it.

Friends also have told me that using a Tivo or other digital recording technology changes the entire television experience, making it much more enjoyable.  I suspect that's true.  If I had something that would keep track of what I like to watch and automatically record it for my later perusal, I might be inclined to spend more time with the TV.  But having looked at the schedule recently, I don't see that there's enough interesting stuff there to justify the $100 or more per month that cable plus Tivo would cost me.  From where I'm sitting, television looks like a vast wasteland filled with "reality" shows, Desperate Housewives, and advertisements trying to sell me things that I neither want nor need (including other TV shows).  Gag me with an eating utensil.

One result of not watching television is that I'm becoming even more of a social outcast than I was before.  It's surprising how much of daily office and friendly conversation revolves around television shows, be it "reality" TV, sitcoms, or sports.  When the subject of conversation turns to television, I just turn and walk away.  Even before I completely stopped watching TV, I didn't watch most of the popular shows, and the only sporting I follow at all is the Tour de France.  If that makes me an odd duck, that's okay.  At least I'm not sitting on the couch letting my brains leak out my ears.

I spent a lot of time reading and studying the 9/11 Commission Report that I picked up at Borders back in November (see November 25).  At 450 printed pages, plus over 100 pages of notes, it's a huge amount of information.

The Report consists of 13 chapters.  The first chapter starts on the morning of September 11, 2001, and describes what we know of the hijackers' actions, the actions of the flight crews, FAA controllers, passengers, and government officials throughout the day.  Chapters 2 through 8 trace the creation and evolution of "The New Terrorism" in general and al Qaeda in particular, and outline what investigators have been able to put together regarding how and when the attacks were planned.  Chapter 9 describes events at the World Trade Center buildings after the crashes, focusing primarily on the problems that first responders had communicating with their own units and among different units.  Chapters 10 and 11 discuss the aftermath and lessons learned.  Chapters 12 and 13 explore options and make recommendations about what to do to prevent future such attacks and how to restructure government in order to effect those changes.

I was surprised by the thoroughness of the Report.  I was less surprised by what appears to be a completely non-partisan feel to the writing, although I was somewhat annoyed in a few places where the authors were a little heavy handed with their political correctness.  All things considered, though, the Report is quite a good read and appears to present the information fairly.  I'll post my thoughts on the Report here over the course of the next week or so.

I thought that "permanentize" was bad.  The authors of the 9/11 Commission Report managed to include not one, but two that are worse.  In the same sentence!  In Chapter 11, when discussing the failure of imagination that prevented us from envisioning anything remotely resembling the attacks that occurred, the authors write:  "It is therefore crucial to find a way of routinizing, even bureaucratizing, the exercise of imagination."  Routinizing?  Bureaucratizing?  What's wrong with "It is therefore crucial to find a way to make the exercise of imagination by bureaucrats a routine occurrence."  Of course, when you can understand what they're saying, you can see that what they're asking is impossible.

Such is the nature of government reports.  Still, it's a good read if you're interested in that kind of thing and don't mind a fair amount of bureaucrat-ese.  Cautiously recommended.

I spent a little more time on the date and time parsing code that I discussed on December 17.  I have created a .NET structure, called W3CDateTime, which maintains the time and UTC offset.  The class can parse RFC822 format dates as well as W3C date time values.  The structure interface mimics the interface for System.DateTime, including the ability to subtract dates to get a TimeSpan value, to add TimeSpan values to dates, and to compare the date values.  It includes all of the System.DateTime functionality except the internationalization options for input and output and some of the date format conversions (OLE Automation and File Date Time).

Creating the structure was an interesting study in the use of regular expressions and interface design.  It also took a whole lot more code than I expected, mostly because I had to implement a whole lot of simple properties and small methods that take three lines of syntactic sugar for a single line of functionality.

I discussed the development and implementation of the structure, complete with source code in C# and Visual Basic in my .NET Reference Guide column.  The full article is here.