The Austringer

I have two new trouble tickets with Verizon FIOS as the connectivity situation continues to be nearly completely non-functional, as it has been since January 10th. The one entered from the Verizon Business FIOS side of things is TXP08R8CY. During the hour-and-a-half tech support conference call needed to get that one going, I happened to inquire about my previously-entered trouble tickets, and was told that they had been closed. Since the problem continues, I insisted that the tech set one up for my residential FIOS account, too. That one is FLCP08R8EN.

If you are a Verizon customer and have difficulty getting through to this site or other sites I run, be sure to reference the above tickets when you put your complaint in. Thanks.

Viewed 2784 times by 551 viewers

I have a bit more information about the connection difficulties I’ve been having with my ISP, Verizon FIOS. I have a residential account in Palmetto, FL with Verizon FIOS. Mostly, it works fine. I can get to a host of web sites without difficulty, and the transfer speeds are great.

I do remote system administration on two servers in the Dallas-Fort Worth Metroplex. Those servers get their connection via a Verizon FIOS Business plan link. (Yes, Verizon, the servers are on an account where serving is usual and expected.) One server provides my regular email, the other serves a whole bunch of web sites via virtual hosting. And things there are mostly working, where the outside world can merrily get pages served on demand.

But…

As of sometime early last Tuesday morning, January 10th, Verizon FIOS stopped reliably talking to Verizon FIOS. I can tell the approximate time of the outage as the last email message my computer here picked up from the server there was at 1:09 AM CT. The problem is very likely to have manifested within a very few minutes after that. And the problem’s characteristics are just plain weird. One expects most ‘problems’ with connections to be user error. Certainly that’s the primary basis of Verizon FIOS’s residential account tech support, who are ready to quit if the problem isn’t solved by having the user clear their browser cache or resetting the router. This problem, though, is more complex and is not localized to my particular account. First, not all connectivity is gone, just *most* connectivity. I can use SSH to log in remotely and use commands that return small amounts of information. Once I try a command that would return a page or more of text, the connection drops with a ‘Broken pipe’ message. There’s a web page that is static and is only a few hundred characters in size that I can successfully retrieve. But none of the web sites that rely on web applications (Drupal, WordPress, and IkonBoard) do anything but spin forever while the browser displays ‘Waiting on …’.

So let me jot down some things I’ve learned about this so far.

* It isn’t a DNS issue, as ‘nslookup’ finds any of the domain names and returns the correct IP address quite rapidly.

* It isn’t a single port failure. Ports 22, 25, 80, and 587 are, at a minimum, included in the affected list.

* It isn’t a complete break, as connections on the scale of a single packet of data at a time work.

* Using traceroute for other websites shows three hops taken within the Verizon routing center in Tampa. Traceroute for the affected servers shows two hops taken similarly, but the third times out.

* My parents live in Lakeland, Florida, a goodly distance away from where I live, and have Verizon FIOS as their ISP. I visited there this past weekend and asked my dad if he had been able to check this blog recently. He said no, not for about the past week. I tried traceroute from their connection, and it behaved the same way as from my home connection. The problem is not localized, it affects other Verizon FIOS customers.

* I’ve heard from Texas where another Verizon FIOS user of the email system cannot connect to the email server. I don’t have a traceroute result from them to compare.

I have two open tickets on this problem with Verizon, FLCP08NT6J and FLDQ090SXY. There are some other people who have posted to the web saying that they are having network difficulties with Verizon FIOS in the same time frame, but I haven’t seen a report that exactly matches what I am seeing. I’m writing this post by the expedient of using a proxy for my browser, which is a nuisance. (While it is on, my Google search results tend to come back in German, which I can’t read.) It’s a bit of a Catch-22, since I’d like to get feedback from Verizon FIOS users, but if the problem is of the nationwide scale that I expect it is, this post will be unaccessible to them from that account. On the other hand, if it is accessible via Verizon FIOS elsewhere, that would be useful information to have. If you are a Verizon FIOS user, I would appreciate it if you could run traceroute from the Verizon account to baywing.net and copy the results into a comment here. I’ll copy my traceroute results into a comment here shortly.

How to invoke traceroute:

Under Windows, open a command prompt. In the command prompt, type in the following:

tracert baywing.net > tr_baywing.txt

It will take a few minutes to complete if you also have the problem I’m having. The result ill be in a text file, ‘tr_baywing.txt’, in that directory. Copy and paste the text in a comment here if you aren’t seeing the problem, or contact me if you are having the problem.

On Mac or FreeBSD, open a terminal window. At the command prompt, type in:

traceroute baywing.net > tr_baywing.net

On Ubuntu Linux, open a terminal window. At the command prompt, type:

tracepath baywing.net > tr_baywing.net

Here’s my email, if you can’t leave a comment here (remove spaces and convert to symbols as indicated): w e l s b e r r at b a y w i n g dot n e t

Viewed 14991 times by 1257 viewers

My connection to the servers in Texas from my home systems is unreliable. For the moment, my only reliable link to various of my web sites and my usual email is via my Android phone. Fortunately, I’m grandfathered into an unlimited data plan and have a Bluetooth keyboard. But that is still not a long-term solution. I have a trouble ticket in for my Verizon FIOS ISP that has been active since this past Wednesday without resolution. I just got a call from Marc saying that another email user is having much the same connection problem, so he’s also putting in a trouble report from his side. The servers run on a Verizon FIOS business plan, so connection outages are a concern on that basis, too.

Viewed 16916 times by 1270 viewers

Some time back, I mentioned getting data off CD-ROM and putting it on hard disk with a second hard disk for back-up. As time passes, this gets more critical. I think archivists start getting antsy about CD-ROM after a decade or so, and I have media that go back to 1996.

And I have run into CD-ROM data disks with various reading errors.

So I thought that I would mention a freeware tool for Windows that addresses getting what can be gotten from a CD-ROM with problems. This is Roadkil’s Unstoppable Copier (RUC). Fortunately, you can stop it in bad circumstances by killing the process in Task Manager. I’ve done this after setting it to work on a CD-ROM with an obvious, visible blemish. In its default setup, RUC will attempt multiple reads of bad sectors in order to recover as much of a file as possible. This leads to it taking a long, lllllooooonnnnngggg, time to get through a patch of damage. Longer than I was willing to wait, anyway. So in the “Settings” tab, I set it to “Auto Skip Damaged Files”. This copies off all the undamaged files from the CD-ROM, and it does so fairly expeditiously. For some CDs, I may decide to let it trundle for a few days to analyze things, but first I want to get as much of the good stuff secured as I can. This tool looks to be a help in that regard.

The lengthy recovery process is probably most useful for large text files, where recovering a majority of a file is preferable to losing all of it due to a possibly small section that is damaged. For binary files, this may not be universally useful. The data files I have are raw integer data, so as long as the reconstituted file preserves the same length, I can recognize the bad patches and leave them out of analysis. That may not hold true for ZIP files and other compressed archives, JPG images, and the like.

Viewed 95917 times by 4734 viewers

It’s been a long time coming, but the paper on evidence for multiple sound sources in the bottlenose dolphin appears in the October 15th issue of the Journal of Experimental Marine Biology and Ecology. I’ve been told that the PDF will be freely available soon, hopefully in the next week or so.

The abstract is:

Indirect evidence for multiple sonar signal generators in odontocetes exists within the published literature. To explore the long-standing controversy over the site of sonar signal generation, direct evidence was collected from three trained bottlenose dolphins (Tursiops truncatus) by simultaneously observing nasal tissue motion, internal nasal cavity pressure, and external acoustic pressure. High-speed video endoscopy revealed tissue motion within both sets of phonic lips, while two hydrophones measured acoustic pressure during biosonar target recognition. Small catheters measured air-pressure changes at various locations within the nasal passages and in the basicranial spaces. Video and acoustic records demonstrate that acoustic pulses can be generated along the phonic fissure by vibrating the phonic labia within each set of phonic lips. The left and right phonic lips are capable of operating independently or simultaneously. Air pressure in both bony nasal passages rose and fell synchronously, even if the activity patterns of the two phonic lips were different. Whistle production and increasing sound pressure levels are generally accompanied by increasing intranarial air pressure. One acoustic “click” occurred coincident with one oscillatory cycle of the phonic labia. Changes in the click repetition rate and cycles of the phonic labia were simultaneous, indicating that these events are coupled. Structural similarity in the nasal apparatus across the Odontoceti suggests that all extant toothed whales generate sonar signals using the phonic lips and similar biomechanical processes.

This was a big undertaking, requiring the coordinated effort of a lot of talented and busy people.

Diane Blackwood designed and implemented our acoustic recording layout and the dolphin stationing device and biteplate, and made sure the amplifying equipment was operational and protected from incident. (Incidents with electronics in proximity to sea water are all too common.) I designed and wrote the software that acted as a multichannel digital data recorder, the data reduction program, and the analysis program. Bill van Bonn was our veterinarian who spent our data recording sessions lying prone on the dock as he placed, checked, and positioned the endoscopes and pressure catheters. Our principal investigator, Ted Cranford, operated the video side of things, including the high-speed video capturing the endoscope views. Sam Ridgway and Don Carder consulted with us, helping us with the use of the pressure catheters (which had previously been used in two prior studies they authored). Monica Chaplin and Jennifer Jeffress were the dolphin trainers on the spot during data recording. Tricia Kamolnick and Mark Todd were trainers who helped get the subjects prepared for our data collection process, and Mark Todd implemented the regular video system. It took between two and three hours each data collection day for us to set up, test, and calibrate all the equipment. Breaking down took somewhat less time, but I would still have to run a custom program to demux the data, produce images visualizing the data for each trial, and then shift the day’s data off the hard disk and on to CD-ROM media.

Update: The Marine Mammal Center has put up the PDF of the paper.

Viewed 111159 times by 6083 viewers

I jumped into discussion of a comment by Greg Laden on Facebook that touched on the Turing test.

There was a comment by Dan Fincke that got me interested:

indeed, at this point I’m generally more impressed when I’m convinced a girl talking to me online is NOT a robot

My reply:

Dan,

Ironically enough, the Turing test as presented by Turing in his famous paper is not as generic as most people think. A male observer compares the conversation coming from a female correspondent and a program, and is supposed to pick out which is the female. So your comment fits right into Turing’s original test conditions.

One can speculate that Turing’s own gender identity issues had something to do with him casting it in that way. The difference between the somewhat-mysterious other gender and a program trying to imitate a female may have been considered by Turing as a more difficult for a male observer, and thus a slightly lower bar as a sufficient criterion for intelligence in a computer program. Or it could be that he just forgot to clarify that the imitation game with the computer involved could be gender-neutral if we wanted to do so.

Greg Laden jumped back into the discussion:

Dan, given a recent conversation on facebook, I’m impressed when a girl talking to me on line is NOT some guy who is pretending to be a girl but really looks like ZZ Top.

Wesley, interesting. I think but I could be wrong, that most of the first order (or should I say first generation) descriptions of the Turin test do not say that. I’m not sure if I’ve ever read the original (I think the first place I saw it was in something written by Gardner).

And my response:

Greg, you are right that most descriptions of the Turing test after Turing are phrased more generically. In the original paper, Turing says:

The new form of the problem can be described’ in terms of a game which we call the ‘imitation game’. It is played with three people, a man (A), a woman (B), and an interrogator (C) who may be of either sex. The interrogator stays in a room apart from the other two. The object of the game for the interrogator is to determine which of the other two is the man and which is the woman. He knows them by labels X and Y, and at the end of the game he says either ‘X is A and Y is B’ or ‘X is B and Y is A’. The interrogator is allowed to put questions to A and B thus:

C: Will X please tell me the length of his or her hair?

Now suppose X is actually A, then A must answer. It is A’s {p.434}object in the game to try and cause C to make the wrong identification. His answer might therefore be

‘My hair is shingled, and the longest strands, are about nine inches long.’

In order that tones of voice may not help the interrogator the answers should be written, or better still, typewritten. The ideal arrangement is to have a teleprinter communicating between the two rooms. Alternatively the question and answers can be repeated by an intermediary. The object of the game for the third player (B) is to help the interrogator. The best strategy for her is probably to give truthful answers. She can add such things as ‘I am the woman, don’t listen to him!’ to her answers, but it will avail nothing as the man can make similar remarks.

We now ask the question, ‘What will happen when a machine takes the part of A in this game?’ Will the interrogator decide wrongly as often when the game is played like this as he does when the game is played between a man and a woman? These questions replace our original, ‘Can machines think?’

Turing does himself later refer to the conditions of the game he introduced more generically, which probably licenses everybody else to treat the test as gender-neutral.

It might be urged that when playing the ‘imitation game’ the best strategy for the machine may possibly be something other than imitation of the behaviour of a man. This may be, but I think it is unlikely that there is any great effect of this kind. In any case there is no intention to investigate here the theory of the game, and it will be assumed that the best strategy is to try to provide answers that would naturally be given by a man.

But I still find it interesting that Turing’s explicit description of the “imitation game” is a *gender-discrimination* test, even with the computer in play.

(I’ll note here that my recall of the original did not include the note that the observer could be either male or female, and that vitiates part of my speculation from the first comment I made above.)

Viewed 124301 times by 8318 viewers

This New Scientist article discusses some really cool results coming out of the Devolab at Michigan State University. In for particular attention was my colleague, Laura Grabowski, who defended her dissertation on memory evolving in Avidians shortly before I left MSU. She is now a professor at the University of Texas – Pan American in Edinburg, Texas, continuing her work on artificial life.

Rob Pennock and Jeff Clune also got attention in the article, and a paper of mine (with Laura and Rob) published last year got a link in the article.

Viewed 52069 times by 5588 viewers

I heard a segment on NPR this evening about the Toyota sudden uncontrolled acceleration problem (I’ll just call it SUAP). They were following the lead of the Wall Street Journal, who said:

The U.S. Department of Transportation has analyzed dozens of data recorders from Toyota Motor Corp. vehicles involved in accidents blamed on sudden acceleration and found that at the time of the crashes, throttles were wide open and the brakes were not engaged, people familiar with the findings said.

The results suggest that some drivers who said their Toyota and Lexus vehicles surged out of control were mistakenly flooring the accelerator when they intended to jam on the brakes. But the findings don’t exonerate Toyota from two known issues blamed for sudden acceleration in its vehicles: sticky accelerator pedals and floor mats that can trap accelerator pedals to the floor.

What the WSJ reported, though, doesn’t exonerate Toyota of anything.

NPR had a commentator on who said something to the effect that 100% of the cases examined showed the same thing, and that one would be hard pressed to argue that the computers got it wrong every time. Not at all, Mr. Non-programmer dude on the radio; all it shows is that the fault is upstream of the black-box recorder and not downstream of it. And it isn’t just the driver who is upstream; there is a lot of Toyota software and hardware there, too. If the Toyotas have an intermittent fault that causes the brake to be recognized as if it were the accelerator, it would explain the data far better than the “all those drivers forgot which pedal is the brake pedal, some of them for minutes at a time” conjecture. That’s just one way in which the problem might occur. In any case, it appears that the data recorders do tell us what the computer controlling the car operated upon, which is full-throttle acceleration and no attention to brakes whatsoever, which corresponds neatly with the survivors’ reports of what happened to them.

I’m thinking when all is said and done, this is going to be discovered to be a software fault in Toyota’s control program. I’m hoping the commentator on NPR gets 30 seconds of airtime to make an abject apology to the survivors when that happens.

Update: I found the NPR All Things Considered transcript, and the fellow whose name I didn’t recall is Mike Ramsey of the Wall Street Journal.

NORRIS: How many data recorders were analyzed? And of those, how many of these accidents were found to have been caused by driver error?

Mr. RAMSEY: Well, we have been saying several dozen, all of them that were -fit the criteria, were found to have the brake not depressed and the accelerator wide open. So 100 percent of the incidents where it fit that criteria, that’s what was found.

NORRIS: One hundred percent?

Mr. RAMSEY: Yes.

NORRIS: It sounds like, upon hearing that, that the government might be on its way toward exonerating Toyota.

Mr. RAMSEY: Well, when it comes to incidents where people are claiming electronic throttle control, the government has already said they have no evidence of it. This set of data, what it does is it completes the other side of it, which is if it’s not that, then what is it, right? It’s probably driver error. So the government has been hesitant to say that so far.

[...]

I totally understand the position of these people. And if you hear many of these anecdotes, it’s incredibly compelling to hear them and all of their evidence. That said, when you have dozens of incidents that are similar where people say they were stepping on the brake and the car accelerated anyway and hit and that all of these incidents show virtually the same findings, that’s difficult to believe that the computer was wrong and, you know, they had a special instance.

(Emphasis added.)

Mike, the data recorder can say what it says and the survivors still be right. Try doing some embedded programming sometime. You haven’t come up with anything that in the least puts their accounts in a bad light, at least not to those who know something about computer control systems.

And be scripting your apology.

Update 2: I’ve marked in bold a particularly interesting piece of information from Ramsey. We have dozens of incidents that show exactly the same thing: no depression of brakes ever, and full depression of the accelerator throughout. This pattern is not what one would expect of humans behaving either in panic, where accidental touching of the brake would be likely, or in Mr. Ramsey’s alternative of confusion of pedals. Pumping the brake is common, so if people were confusing the accelerator with the brake, we’d expect to see some fraction of those incidents showing variation in the accelerator control, and according to Mr. Ramsey, we never see that. That’s pretty damning for Toyota, I think. Having absolutely the same data pattern across dozens of drivers when some of those incidents went on for a significant amount of time doesn’t speak to mass confusion of drivers; it says “computer screw-up” to me.

Viewed 59714 times by 6077 viewers

For a while after moving in here and getting our new ISP, we were able to send our email through our server in Texas using port 25. That stopped working, so it was time to deal with the joys of managing with an ISP blocking port 25.

The first step was getting Postfix on our email server in Texas to use the submission port, port 587. There’s about six lines in Postfix’s “master.cf” configuration that have to be uncommented and restarting Postfix, plus making sure /etc/services has port 587 uncommented.

I tested things out using my Thunderbird email client, and things went fine, with just a dialog about accepting the SSL certificate from the email server. That made me feel good.

Then I tried to get Diane’s antique installation of Eudora to connect up. My mood went down. Trying to add “:587″ to the SMTP server name resulted in Eudora not figuring out where the server was, despite various places online where Qualcomm says appending “:587″ would fix things up. Another round of searching turned up an odd procedure: copy “esoteric.epi” up to the main Eudora directory, restart Eudora, then set the port for SMTP in the new “Ports” section of the Options part of the menu. That brought me to the next stop: SSL negotiation failed because the certificate had expired. Last year, Jeff handled getting the certificate set up, so now I got to work on the SSL certificate. But things did eventually fall into place, and our email now flows in its accustomed channels once again.

Viewed 57060 times by 5471 viewers

Following up on a comment from Dick Hoppe, I expanded upon the data compilation I wrote about earlier concerning the Manatee County 2010 Tax Certificate Auction. Now I’m pulling in data from three additional pages and have it all tidily summarized in the resulting comma-delimited CSV file. I made a short demo CSV file with three of the entries so people could pull it into a spreadsheet and see how it works. I made a page to explain what I had and why an investor ought to want to have it here, and that includes PayPal links for people to pick either the MS-DOS/Windows or the Unix/Mac OS X version.

My biggest problem is there is a small market for this, and I don’t really have a good way to make them aware that there is an alternative to them doing all their information look-ups manually themselves. I tried making a posting to Craigslist, but all the responses I’ve gotten so far are spam.

Anybody else have experience with time-limited, targeted market information compilation marketing?

Viewed 51755 times by 4581 viewers

Manatee County offers tax certificates to bidders. When property owners fail to pay their taxes, and that is happening a lot right now, the county gets other people to pay the taxes and gives them a tax certificate, which is a lien against the property. Each year, an auction happens where people can bid to get these. The bid amounts are in percent interest, and range from 18% at the high end down to 0%. The person bidding the lowest percent interest gets the tax certificate, after, of course, they pay the county the outstanding taxes.

Today, there was a practice auction. This is all handled online now. The page included the option to download data on the 9,000+ properties in CSV, XLS, or XML formats.

Diane is interested in the process and specifically in the land just to the south of our property. It currently has unpaid taxes, and if the executors of the former owner’s estate don’t pay up by June 1st, it will be included in the tax certificate auction. But she is also interested in what else is available out there.

That brings up an interesting problem. The downloaded data is minimal, giving just a parcel ID, outstanding tax balance, and some auction-related attributes. On the other hand, Diane would like information that is available online from another county office, that of the Property Appraiser.

I worked on a Python script to handle the job of getting additional information on acreage, zoning, the address, and bits like that. I hadn’t done anything with Python regular expressions to date, and started looking at that and getting less enthused by the minute. The issue is getting data out of an HTML page downloaded from the Property Appraiser. I could have it done in Perl right offhand, but wanted to develop my Python skills a bit.

On the other hand, getting the job done is the top priority, so while looking stuff up, I ran across the BeautifulSoup module for Python. The web site sounded promising, and a number of other people seemed to have found it useful. Very useful.

BeautifulSoup is an HTML/XML parser. It aims to not only handle clean XHTML, but also to do reasonable things with the sort of HTML people were writing when the Web was young, in other words, bad HTML.

I downloaded the module distribution, and got it uncompressed. Setup is simply

python setup.py -install

My usage so far is to pluck values out of adjacent cells in a table. I can load a BeautifulSoup object with the HTML in question, then ask it to find the label I’m looking for in text. Then I just ask it to retrieve the next text in the document, and that is the stuff I’m looking for.

Anytime one gets started with a library to do a job, it can take a while to get going with it. BeautifulSoup let me get my job done without a lot of effort on the initial learning curve. Right now, my script is about halfway through getting the additional data wanted for those 9,000+ properties. We’ll be able to look it over in the morning. The whole script I’m using is less than a hundred lines of code, and that reads in a CSV file, traverses that, gets the associated profile page from the Property Appraiser for each property, parses that with BeautifulSoup, adds the additional fields of info to the original, and writes out a new CSV file with the more complete data set.

Viewed 53952 times by 4982 viewers

We got our new ISP activated on Saturday, and we had selected Verizon FIOS. On a dollars per bandwidth unit basis, it was by far the most effective way to spend the money. The choices where we are were Verizon DSL, Bright House cable, and Verizon FIOS.

I had priced the DSL a couple of months ago, and Verizon was offering 1 Mbps for $19.99/month and 1.5 Mbps for $29.99/month. We were considering the 1 Mbps DSL service simply on the cheapskate basis. However, when I checked again last week, the prices had been sharply changed upward. The 1 Mbps was $29.99/month, and the 1.5 Mbps was $39.99/month. I happened to have a chat session going with a Verizon representative, and part of it went something like this:

Me: So what additional value has been added to the DSL options to make them worth $10 more a month now than back in January?

[2 minute pause]

Verizon Rep: I’m sorry, I don’t have any information available about that.

Me: Good answer.

While we didn’t really want to reward Verizon for the predatory pricing structure they’ve created on DSL, the bandwidth available with Verizon FIOS was just too tempting. The FIOS Internet service starts at 15 Mbps downstream and 3 Mbps upstream at $54.95/month. It’s more than we wanted to be budgeting for our internet, but we really do use it.

A Verizon service person called last Thursday to discuss access to our driveway. It’s a mere 663′ long. His job was to get the fiber optic cable laid down to the house. We found out that they had to put in a splice; they’ve marked that patch of ground with flags and recommended that we don’t plan to extend our driveway over that spot. I had informed them about the long driveway in the chat session, and they get their fiber optic cable in 1000′ lengths, so they should have had plenty to manage to get there without a splice.

The actual install went fairly smoothly. Verizon says installation may take between four and eight hours, but our install was done in about three.

I did a Speakeasy bandwidth test, and the gear delivered a bit more than advertised, so that’s to the good. We’ve been using Bright House cable to access the internet since last August, and we’ve had a variety of annoying lapses in what we’ve been able to do. For instance, we use email on a server located in Texas. We have not been able to send email through that server for several months. That has now been remedied.

The next step is to get our home internal network set up again. Right now, FIOS does look like it will help us get done the things we need done on the internet.

Update: OK, I found a slightly annoying thing here: poor DNS resolution. Apparently, the FIOS router defaults to a set of not-so-hot nameservers. Fortunately, I can specify better ones on my individual computers. See this page for an explication of the problem and the fix.

Viewed 49001 times by 5359 viewers

I have data on Compact Disks (CDs) from past projects. The technology was getting toward being affordable around 1996. CD writers dropped under $100 for the first time somewhere around there, and media started selling for less than $5 a disk. The amount of storage space on a CD was comparable to the size of hard disks available at the time, and optical storage seemed far better than tape as a medium. So now I have cases, drawers, and spindles of CDs dating right back to 1996.

No storage medium is perfect, so archived data is a commitment and not just a static collection. Last month, Sam asked me what I would like for my birthday. I said I wanted a disk for backing up data. After having a look at off-the-shelf external hard drives, it seemed that all the models I looked at had warranties of 1 year or shorter. However, if you buy an internal hard disk and a separate USB enclosure, the warranty on the drive can be much, much longer. Sam and I visited the Newegg site and picked out a Western Digital 1.5 terabyte drive and a Rosewill USB enclosure. The drive comes with a 5-year warranty. I can pair this with another 1.5 terabyte disk so that I can copy off my data from the CDs, then copy to the second hard disk.

Back when I was about to move from California to Michigan, I had a chat with a fellow who works for the Internet Archive. That is a project whose modest aim is to store the World Wide Web. All of it. You can browse sites as they were in 1995. Well, with a few caveats. My acquaintance said that the Internet Archive’s data storage was based on consumer-grade IDE drives. You can get them cheap and in quantity, and if you store things on multiple disks, the redundancy will help. That’s because disks fail. With an organization like the Internet Archive, they rack up lots of failures. They have to be swapping out bad drives and attempting to restore content from remaining copies on other drives. And they couldn’t, he said, quite keep up with the failures. Some data does get lost because failures occur before the redundancy can be exploited to restore some sites.

I figure for my purposes, the data I have is a copy of what my colleagues have, and for the hard disk copy, I aim to have two of those. I think that should be sufficiently paranoid. The process or workflow takes about six to seven minutes per CD to create a directory, copy the files, and mark the CD as copied. I’m working on the third page out of 32 pages in a CD case now. This will take some effort, but then I invested years of my life getting that data in the first place.

Viewed 35432 times by 6642 viewers

Apple announced its iPad tablet computer today. The device seems to be mostly a large-screen iPod Touch. The intriguing aspects of the iPad, at least to me, were that Apple says that for the 3G versions ($130 extra over the WiFi-only versions) these devices will be unlocked, and that Apple has arrangements with textbook publishers for EPUB content. It seems that Apple was able to wring some few concessions from AT&T concerning the unlocking and the two tiers of data plans. While the data plan costs are not cheap, they manage not to be exorbitant.

I saw that some other commentators were perplexed about the time taken in the announcement to show Apple’s iWork applications as they are ported to the iPad. I think, though, that a major market for the iPad might just turn out to be among high school and college students. Consider the points made and that market:

- Light enough to carry around in the backpack (If a student can skip carrying even one textbook and carry an iPad instead, they will be lightening their load.)

- 10 hour battery life, good enough for the school day

- Low cost applications that will be good enough for note-taking and in-class analysis

- Capable of holding and displaying full textbook content in color plus supplemental multimedia

- Cost low enough that it is compatible with current budgets for textbooks

- WiFi for on-campus connectivity and research

The fact that it also does a bunch of multimedia service plus gaming will be seen as a plus, at least by the students if not their parents.

Viewed 30658 times by 6433 viewers

William Dembski and Robert Marks finally managed to turn one of their joint manuscripts into a publication. The paper will appear in IEEE Systems, Man, and Cybernetics. There is a PDF of it available here. I’m in the midst of packing, so I just confirmed that Dembski and Marks carefully preserved the error I informed Dembski of almost 9 years ago and Marks almost 2 years ago.

I mentioned some time ago that I would write a response for publication, and I intend to do that. Right now, though, the trailers are partially loaded and there’s a fair bit more work and the trip to do yet.

One more thing… Dembski wants this paper to count in the pro-ID peer-reviewed category and show up in the DI list and whatnot.

P.S. Our critics will immediately say that this really isn’t a pro-ID article but that it’s about something else (I’ve seen this line now for over a decade once work on ID started encroaching into peer-review territory). Before you believe this, have a look at the article. In it we critique, for instance, Richard Dawkins METHINKS*IT*IS*LIKE*A*WEASEL (p. 1055). Question: When Dawkins introduced this example, was he arguing pro-Darwinism? Yes he was. In critiquing his example and arguing that information is not created by unguided evolutionary processes, we are indeed making an argument that supports ID.

The only way to understand the above is if one accepts the religious antievolution “two model” way of thinking. That goes like this: there are only two alternatives, evolution or {creation | design}. Therefore, evidence against evolution is evidence for {creation | design}. The “two model” argument got well-deserved thrashings in McLean v. Arkansas and Edwards v. Aguillard. It’s nice to see Dembski continuing to stick with just the classic, long-rebutted religious antievolution arguments.

Viewed 36841 times by 6925 viewers

I retired my banner that I put together in my hospital bed in 2004 and have set up a set of new banners that get picked randomly with page requests. The original aspect ratio was just too long at 8.84:1, so I shifted it to 8.84:2.

GIMP provides a selection tool for a fixed aspect ratio, which was just what I needed. Rotate, crop, scale, apply levels, unsharp mask, and I can save off another banner image. I’ll try to add more to the mix as time goes by.

Viewed 24809 times by 5943 viewers

Over the weekend, I vacuumed out my video editing desktop machine. It’s been a while since I used it, and it had collected its fair share of dust.

The machine is based on an Asus P4B motherboard and 1.8GHz Intel P4 CPU. This was state-of-the-art when I built the system in 2002. It was built around the requirements of the Pinnacle DV 500 DVD video card. This is a very picky piece of hardware. It only works with a limited subset of motherboards and only has drivers for Windows 98/NT/2000/XP. OK, given all those requirements, what does it offer? It does analog video signal digitization as well as Firewire for DV capture. And it will provide real-time preview for various supported video transitions when paired with Adobe Premiere.

When I was putting the system together, it seemed that hard disks over 40GB were particularly prone to failure. I had gone with a 20GB boot drive and a 40GB data drive originally. A couple of years later, the rough patch in hard disk QC seemed to have passed, and I replaced the 40GB with a 120GB drive. And there the system remained from about 2005 to this past weekend.

Given that Diane and I still are slowly working off a substantial load of debt from grad school and my health issues (see earliest posts here from 2004 for the grisly details), there is now no discretionary budget for computer gear. We get what is needed when it is needed. Our purchases since 2003 include a laptop bought as a replacement under insurance, a $120 laptop bought from surplus at Diane’s college to replace her failing Dell, miscellaneous hard disks for our files, and a desktop upgrade almost completely underwritten by a generous donation from one of my readers here. We are still using daily a desktop computer bought used around 2002.

So doing anything to the video machine had to fit two constraints: I still needed to be using WinXP for the OS, and it needed to cost nothing but a bit of my time. I located two hard disks that had been replaced in other systems by higher-capacity drives, a 60GB and a 200GB drive. I used a partition cloning tool to copy over the 20GB boot disk to the 60GB. This would let me install various new software packages, including the Microsoft C# Visual Studio Express development environment, which uses 1.1GB all on its own. I cleared off the 200GB with a new NTFS format to add it as a second data disk.

Why video and why now? My time at MSU is drawing to a close, so we will be moving shortly. I have a stack of video tapes in DV, Hi-8, and VHS formats. I’ve always intended to get these digitized, and between having little working space and not having a good workflow sorted out, it hasn’t happened. It seemed to me that if I got things squared away, I could be doing a basic video digitization and archiving sequence in parallel with other activities. After all, most of the time is going to be tied up in either playing a video source for capture, or rendering captured video to an archival format.

So given the extra space and some sprucing up of the installed system, I’m ready for moving video bits around. My goal is to have the video available for non-linear editing in nearly-pristine condition. Raw DV is too large to be efficient. After asking around a bit, I’ve started with the aim of rendering to a multiplexed MPEG-2 stream as a format that is easy on storage requirements, but loses little of whatever quality there was in a DV source.

This starts with capture, which for my system comes through the Pinnacle DV 500 DVD. I’m starting with the DV tapes, as this promises to be easiest all around. Pinnacle has their DVTools package that does a fine job of capturing from a DV source. I’ve done about four tapes so far with no dropped frames at all. It does, however, continue past the last actual image from the DV source if the tape isn’t completely full.

When the capture is done, I fire up Sony Vegas and put the capture file on the timeline. It doesn’t take long to snip out various unneeded bits, including the extra stuff at the end of the capture file. If it was all related to one event, I’m ready to render that. If the capture file includes multiple events, I save the Vegas project file for the complete thing and then begin rendering each sub-section separately.

Because these are intended for further editing sometime when I have free time, I’ll just be putting these out to DVD as data. I have ImgBurn installed to handle that. There are at least three tapes in there that I will also author a video DVD for that I know of off-hand. Those will take a bit extra effort, but I’m not doing that for most of these.

Of course, suggestions are welcome. Please do remember the constraints I have, so software package suggestions should be for open source or freeware packages. I do have a laptop that dual-boots Vista and Ubuntu, plus a desktop running FreeBSD 7.2, so video processing on those systems could be done if there’s a suitable benefit.

Viewed 15287 times by 4738 viewers

I ran across this cheat sheet while looking for an answer to setting line spacing of single-spaced within paragraphs and double-spaced between paragraphs in the front matter. While it doesn’t have the answer to that, it does look like a very handy reference for more commonly encountered situations in .

If you are wondering what is, it is a document-processing and typography system. It is to a word processor what a process camera is to a point-and-shoot consumer camera. It’s big, has a steep learning curve, but delivers results far beyond what can be done with consumer-grade word-processing applications, or at least makes it possible to do those tasks with far less hair-pulling.

Technically, is a set of macros originally by Leslie Lamport built on the typography system of Donald Knuth. Documents in are actually programs, so the process of building a document in is much like software development. While there are commercial versions of systems, pretty much everyone I know uses free, open source versions like MikTeX or TeXLive. There are a number of frontends that help users construct and typeset documents using : TeXNicCenter for Windows, TeXShop for Mac, and Kile for the KDE GUI on Linux and FreeBSD.

Why use and not either word processing programs like WordPerfect and Word or desktop publishing packages like Ventura and Quark? First, has excellent mathematics typesetting capabilities. It is the sole format accepted by many journals that often deal with typesetting equations. If you are writing for such journals, there is no alternative. If you want to publish math-heavy text and not spend oodles of your time trying to figure out what went wrong in an “equation editor” for a consumer word processing program, you want . Second, it incorporates a huge amount of typography experience. If you are concerned about making documents that are not just formatted well, but make it easy on the eyes of the reader, provides that for you. It is flexible enough that if you think you know better, you can override just about anything, though most of the time that’s not really going to help your readers. Third, automates just about everything that makes writing large documents a hard task. Let me explain that by example.

When Diane and I were writing our dissertations, we had a task of putting together several chapters of material where the final document had to conform to a long set of rules used by the Thesis Office at our university to assure both consistency across dissertations and to allow micro-filming archives to be able to use the result. In particular, there are rules about the placement of figures and tables relative to where they are first referenced in the text. In word processors, you place your text and you place your figure or table, and there is no effective control over where the word processor finally decides to put the figure or table. I had several chapters of material, and I tried WordPerfect and Microsoft Word, without success. I tried Microsoft Publisher and Corel Ventura, but also ran into difficulties there. It was around that time that I started looking at as an alternative. I found that dissertations in the electrical engineering department were often done in and that there was a thesis class (a sort of configuration or environment document setting a style) for that the EE department made available. This was like an existence proof; people had actually managed to get the thesis office to accept their manuscripts when done with the thesis class. I asked Jeff Shallit for recommendations on books, and he pointed me to the book by Leslie Lamport and the Companion book by Goossens, Mittlebach, and Samarin. What I found back in 2002 was that getting acquainted with definitely took some effort, but it almost immediately was paying off. My figures and tables weren’t going hither and yon willy-nilly, they were pretty much where they needed to be, or could be tweaked to do so.

Then, some of the other benefits started becoming apparent. While word processors have some mechanisms for generating front matter (table of contents, lists of figures and tables), could do this in a very systematic way that basically took the entire load off my back. The other bane of the dissertation writer is references. The Thesis Office wanted all references to be cited in a consistent style, to be formatted in a consistent style, to appear in order, and that every citation in the text would appear in the references, and no reference would appear that was not cited in the text. That last one puts a huge load on someone who is organizing a large set of references for themselves. Let’s say that your committee decides that you should remove some text including a citation that only appears in that text. You have to remember not only to remove the text, but to revamp your bibliography so that the now-uncited reference no longer appears. has a helper program, BibTeX, specifically for handling bibliographic data. Using BibTeX and the natbib style, I was able to address all the concerns of the Thesis Office while keeping things pretty simple for me. BibTeX allows you to set up one or more bibliographic source files containing all the references that you might want to use in your document. Within the document, citing a reference occurs using a “\cite” command. There are variants to allow for various in-text citation formats. The cite command is given a parameter that links to a particular reference in one of your BibTeX files. sets up a file used by BibTeX to pull in just the references that are actually used, and BibTeX applies the desired style to produce the typesetting for the references section. The result is that the references section went from something needing a lot of continuing effort to maintain to needing almost no effort to maintain. That sort of assistance is invaluable when what one wants to be doing is writing content and not worrying incessantly about keeping all the effects of changes one makes to the layout in mind.

Something I didn’t use in writing my dissertation that provides is indexing. If you want to produce a large manuscript with an index, this is something that you can do pretty easily in . Basically, as you go along in the text, you place an index tag next to the text that you want the index entry to refer to. will track the entries and the corresponding page numbers for you. If you re-organize your text, say by swapping chapters 2 and 3 around, you don’t have to re-do all those page number references in an index, will handle it for you.

provides several basic document classes for you, and you can find extensions online. The basic ones include “letter”, “article”, “book”, and “slide”. That last allows you to generate presentations in . Then there are all sorts of styles that one can add on. For example, if you want to write screenplays using the standard formatting rules, the screenplay style can help you. (If, though, you are really intent on screenplay writing, you probably want to look at Celtx. [Addendum: Looking a bit more at the Celtx website, I found this: "TypeSet provides precise automatic formatting of your script to industry and international standards. The Celtx server uses the very powerful LaTex typesetting tool to deliver perfectly formatted scripts."])

There’s a system called LyX that places itself in between full and the usual way one uses a word processor. is used by LyX as a back-end, and you get a display of text that is a bit closer to the usual WYSIWYG experience, but cast by LyX as “what you see it what you mean”. Unfortunately, LyX documents are not simply standard , which to me is a limitation of the system.

Since writing my dissertation, I have relied upon for all my serious writing work, save where a collaborator has insisted upon something else. I use for writing letters and it is the basis for the six or so pending article manuscripts I have. My curriculum vitae/resume is handled in , and I have that set up such that I can generate documents of different lengths and detail, plus tuning the focus of my research statement, all by changing a couple of configuration settings. This means that I have one source text whether I want a CV or a resume, or whether I’m sending the result to someone interested in my biology background, my computer science background, or appreciates my interdisciplinary approach. That also means that I can keep things up to date with changes to just one file, and not about a dozen different ones to handle the most common sets of configuration changes that I use.

If you don’t do equation type-setting, don’t need figures and tables to go where you want, don’t need front matter or bibliographies, and don’t need an index, you’ll probably be perfectly happy using the usual word processor. If you do need any of those things, then you owe it to yourself to check out .

Viewed 13786 times by 5909 viewers

The email server I use was having some hardware issues. Marc picked up a new box and disk, and Jeff, who has somewhat more spare time at the moment than I do, suggested we go with Ubuntu Server 9.04 for the new install.

So we switched from a FreeBSD 6.3 box to Ubuntu Server today, and on it the new mail system was Postfix/MySQL/Courier. We spent a bit over four hours copying files and preparing the user accounts to use the new system before bringing the Ubuntu server online in place of the old FreeBSD one.

The rest of my day has been spent in fixing up other issues, like switching over the couple of WWW domains that were served from there and setting up email list software.

I’ve been using Majordomo for email lists since the 1990s. Unfortunately, that’s about the time of the last update for that software, too. So I am getting acquainted with Mailman instead.

Hopefully, most of those issues will be sorted before the end of the weekend.

On a somewhat more personal note, the way that I’ve done email since the 1990s has been disrupted. I’ve used the .forward file in my user account to pipe incoming email into a Perl script I wrote. It uses a whitelist file and a pattern file to sort incoming mail and append it to a file named for the day and with an extension according to the recognized class of email. Most of my email reading has been done using emacs from the command line of an ssh session. Now I’m dealing with using SquirrelMail as a primary interface, at least until I can work out what to do about the setup. I’m looking into Fetchmail, which I’m hopeful may allow me to do much the same thing as I did before, where my script only stuck back into my incoming mail box those items matching my whitelist criteria.

Viewed 9433 times by 3290 viewers

There was no sight-seeing, but I went to Nashville, Tennessee from last Sunday to last Thursday. This was to present at a conference, the IEEE Symposium Series on Computational Intelligence. I had a paper in the Artificial Life session that I presented on Tuesday, and it seemed to me that it went well. The Aritficial Life session ended Tuesday, though, so I was attending papers given in various of the other tracks at the conference.

On Wednesday, I sat down at lunch just at random, and two attendees sat down beside me. I talked mostly with Bob Abercrombie on my right, but at some point he brought his colleague, Rick Sheldon, into the conversation. As we compared notes on our backgrounds, Rick and I gradually came to realize that we had gone to grad school in computer science together and had worked together just afterwards at General Dynamics Data Systems Division.

If that wasn’t strange enough, I attended several talks in the computer security track on Thursday. One of the attendees seemed more than usually animated and clued in, and I decided to join his table for lunch. I noticed his nametag said he was Daniel Ashlock of Guelph University. The name rang a bell, and I spent most of lunch trying to recall where I knew him from. It turned out that we had both participated extensively in the Usenet talk.origins newsgroup in the early 1990s, and we both have listings on the University of Ediacara faculty roll. We had never met before in person, so we got a chance to discuss this that and the other while proceeding to the airport and waiting for our flight times.

Score another couple points for the small world.

Last Thursday, though, bad weather was moving into Nashville. By the time we go to the airport, the temperature was dropping and the sky looked quite dark off to the west. We had gotten through the security checkpoint and had been waiting in the terminal a while when a PA announcement said that a tornado had been spotted in the area, and everyone was supposed to move away from glass windows over to interior walls or into the restrooms. This was the first time I’d been someplace where a “take cover” advisory had been issued for a tornado threat. While the tornado didn’t come visit the airport, the bad weather made hash of the departure schedule there. After a whole series of announced delays, my flight that was supposed to leave at 5:22 PM actually left about 9 PM. Since I had a connecting flight from Detroit to Lansing, that meant that I arrived in Detroit about an hour after my plane had left for Lansing.

I asked the gate agent for assistance as I came off the airplane. I was told that everything would be handled at the station at Gate A43. I was then at Gate A61. So a fifteen minute stroll later I arrived at the station at Gate A43. No one was there. As I stood there trying to figure out what was next, someone did come by, the nightshift agent for Northwest Airlines. Apparently it is Northwest Airlines policy to run their customers through a sequential gauntlet of liars (the gate agent who sent me to an unstaffed location for assistance) and the rude (the wandering agent whose job is apparently to do as little as possible that would actually help passengers). The one piece of useful information I got from the peripatetic and randomly abusive agent was that late-night service was limited to the Northwest Airlines baggage claim office. So I headed there. The staff at the baggage claim area were pleasant enough, but given that “weather” was down as the reason for the missed connecting flight, they only needed to reschedule me on the next available flight … which would be the following afternoon. They could get me a discounted rate at a hotel, but that was it as far as doing anything to assist me. So I checked the car rental places, figuring that if I could get a car rental cheaper than the hotel, I’d still be ahead. Out of about nine places, only six answered the phone at 12:30 AM, and of those, only two had cars to rent and would provide a daily rate quote, and both of those were over $99.

So around 1 AM I called Diane and asked her to book me a seat on the next Michigan Flyer bus, which would be a 6 AM departure. I didn’t see much point in doing the hotel thing for what would be about three hours of sleep. So I got a seat at door 402 at the terminal, which is where the Michigan Flyer would be coming. I settled in to do some programming and passed the time with that and naps. The Detroit Airport, like the Nashville Airport, offers the Boingo WiFi hotspot service, allowing people willing to part with $10 to hook up to the internet while they are in the airport. Since that didn’t include me, I just worked on things that didn’t need online access.

Eventually, 6 AM came around, and so did the Michigan Flyer. I got aboard, and got to wait some more for whatever paperwork the bus driver found necessary to do. We got moving around 6:30 AM, and had our first stop about fifteen minutes later at the other terminal. After another round of paperwork, we got moving again. There was a stop in Ann Arbor, and another in Jackson. We arrived in East Lansing about 9:30 AM. Diane came and picked me up. I’ve been off schedule over the weekend. I do hope I re-sync soon.

Viewed 9930 times by 3239 viewers