According to Dr Dave there is a serious security flaw in all versions of WordPress. His advice is to immediately disable the “anyone can register option” (Go to the admin panel, under Options->General, about half way down the page) to protect yourself in the meantime.

He is not official WP personnel, but from previous reading around the ‘net, he’s clearly a well respected player. I don’t get many registrations in the first place, so I’ve gone ahead and turned it off myself. I’ll keep y’all posted when I get any more info.

Precomposed versus Combining

In the course of putting together the encodings (called code points) in Unicode, a number of decisions had to be made regarding the current existing encodings, particularly well known and/or well established ones. In some cases, even though the Unicode Consortium has a particular policy regarding some encode vs render issues, there are inconsistent inclusions due to this grandfathering of prior established encodings and (to be quite honest) outright mistakes on the part of the Consortium. The question of precomposed characters versus combining characters is a classic one.

Read the rest of this entry »

Well I spotted this Frazz cartoon today (anyone else think he’s a grown up Calvin from Calvin and Hobbes?)

and it got me to thinking. I had mentioned the Green Card spam to a co-worker of mine and he hadn’t recognized it.

Yikes. I mean, that marked a turning point in the Internet for me. There was never any turning back after that. It’s a little difficult describing the Internet as it was back then to someone today. And don’t get me wrong, I’m not going on about some halcyon Golden Years or something. There’s a lot of good that’s come along with the bad. I absolutely relish the amount of information available online and how I can search for all kinds of things.

At the same time, there are things that I miss. For example, every email that I got was from someone I knew. I’d reply to each and every one of them too, often taking a considerable amount of time and care in composing the reply. People nowadays scarcely know what Usenet newsgroups are (think of there being one single forum, containing all the topics as different threads, freewheeling through the Internet), but again, those were very tightly knit communities by and large. Each one had its own character, of course, but everyone was on their set of groups because they had something to say.

And given that back then the dominant providers were all universities, you had a remarkably high percentage of intelligent and highly educated people on the Internet (not necessarily any nicer; there are classic flamewars I can recall involving the likes of weemba, Richard Sexton, Oleg Kisilev and others — now that’s some well crafted stuff. I mean, these guys were slinging multisyllabic mud ).

Also, because at the time the funding and backbone was provided by the NFS, all commercial activity was prohibited (this ended in 1995) during that time. So imagine a community of university folks, librarians (these guys were big time early information sharers, which is often overlooked), and some government types. Getting online was tough, with tools few and often unfriendly. (”GUI’s? What, are you high?”) So just being online was already an announcement of your personal geekhood. Also, if you weren’t at a university, with the nice T1 connections? I have one sentence for you: 1200 baud rate hell.

The atmosphere was just a bit different then.

Once the commercial spam started, though, the Usenet newsgroups were finished. They took their time dying, of course, and although the carcass may still be feebly kicking, there’s no point in reading that anymore, even though it spawned so much stuff still around today. (Ever wonder where the concept (and acronym) of FAQ came from? Terminology like trolling, spam, smilies (tm for fuck’s sake, now), and so on? Yep.)

Interestingly, the spam moved on to the chat rooms which had been emerging at about this time due to all the private bulletin boards that started to integrate into the Internet. I recall all the various predictions of the Death of the Internet — News at 11 that came with Delphi, with AOL, with Compuserv, with GEnie and so many others’ entry to the Internet. (Yes, the Internet existed and exists as a separate entity from AOL, a fact I find hard to explain to some people these days.) And yet, we absorbed these people, learned to deal with massive numbers of folks who had no clue whatsoever what the Internet was or how it worked. We had culture clashes and swirls and eddies.

In any case, IRC and chat rooms and such boomed, and then fell apart as spambots invaded those too. These days its comment-spam. (For an interesting summary of the history of spam, see this by another person who’s a blast from the past. In particular, I distinctly recall jj’s college fund. I was a denizen of the moria newsgroup at that point, and parodied him by sending out a call for code, so I could finally, at last, someday, complete my program. It was a hit — many people got a chuckle out of it, and did contribute code. One person said “Here’s /* — it may not be much, but you can use it in lots of places!”)

Hmph. I see I’m rambling all over.

What I note is the constant move toward individually controlled entities. My own progression was from the Usenet groups to Listserv mailing lists. I ran a number of these mailing lists, and learned to be quite ruthless with the membership to keep them viable, interesting, and useful for the subscribers. Then I moved on to websites, archiving many faqs that I’d put together in previous years. In the late 90’s I collaborated with several other women on a website that, looking back on it now, was an early blog, back when there was no software for that kind of thing. We’d go in and edit in new things manually, although I wrote a number of utilities for storing our articles in a MySQL database, and another one of us had the idea of tagging our posts (calling them keywords then, of course).

In a way I think blogs are moving toward the old Usenet model as much as possible while retaining the individual control — and the spam’s response is indicative of that, I think. Consider the blog: first most blogs were simply daily updates by their owners, without comments. Then commenting ability was added. Then spam showed up. So anti-spam software developed to help the bloggers control that. Now you see features such as threaded comments, and even utilities such as coComments that help you track all your comments in a single place (very Beta-ish but lots of potential for usefulness), or rss feeds to subscribe to.

Threaded comments and crossposting? These are not new concepts. They’ve been around for decades, mostly on unix machines with difficult to use command line interfaces. That’s something else that I’ve observed over the last fifteen years, too. There were many concepts that were around and implemented long ago, but the interface was too difficult and technical. It wasn’t until the mid 90’s that there was much in the way of graphical interfaces to the Internet available (this is also what spurred on marriages such as AOL and the Internet), but these graphical interfaces started back at ground zero with many of the concepts the comand line interfaces were using! This is why I was such a die hard command line interface user up until about a year or two ago (even now, my work Windows computer functions largely as a station for my ssh connections to our resident unix machine). I simply couldn’t find the level of functionality I was used to, even though the eye candy was so much nicer.

Let me give an example. Until recently, I used the same mail program I had used since I first got on the net (mid 80’s if you must know): the MH mail program (look about half way down for example usage, this is NOT the unix barebones mail program which is all but unusable ). Why did I use this antiquity? Because it was completely programmable. From the first day I used it, I was able to:

1. sort incoming mail in different folders depending on who sent it
2. threaded views, if I wanted (I didn’t appreciate this sort of ability until much later; it’s something that’s best with bulk amounts of stuff)
3. block out mail I didn’t want
4. send out automated messages in some circumstances (eg, vacation, an admin type email)
5. put different emails into different folders based on topics — all email from my advisor in one folder, all email regarding this project into that one, all email I had to answer within the day to another, a sent folder (also sorted out) for my outgoing mail
6. customize it with my own programs — I’d write something to parse it and do something and be able to add it in and have it function with the rest of the stuff. An early example was the retrieval of files I made available to the public via email with a certain syntax in the subject line

Now you tell me which of the modern email programs acquired that level of functionality when? I doubt Outlook is anywhere close to that yet. I was continually frustrated at the extremely primitive level of filtering available. (What do I finally use? Gmail, Thunderbird, and Evolution are all pretty impressive at this point, although I can’t quite customize them the way I used to in MH anymore I find that these days I really don’t need to anyway — there’s lots of other tools that do a better job. If I want to make something available, I’ll just do it over my blog or another website anyway. So there you go.)

Anyway, I have no doubt the next fifteen years will be as interesting. Maybe by then I’ll be sending out holo-casts!

I sometimes think some of my best work will go forever unread. I do enjoy writing. But I’m a programmer, so I spend quite a bit of time with code. Mine, or more often someone else’s. The fun part about the latter can be figuring out why on earth the previous programmer did what they did, why didn’t they comment at all usefully, and how the <censored> does it even work in the first place? And as I pick through the code, sometimes my outrage is vented in the comments I add to the code in order to aid the next programmer, if there ever is one, since after all the code should have been dropkicked to the curb before I even got to it.

So when you see comments like this

/* this program does not actually know about the search_id parameter. it does not care if it is a string, an integer, or purple polka dots from germany. It is simply going to grab it, ignore it, and pass it along to the next program like a good little prole */

you’ll know that I got a tad bit frustrated that day.

My particular favorite example involves a programmer whose job I took over who was capable of churning out astonshing quantities of prose in remarkably short times. But for all the voluminous output, the documentation was rarely actually useful. Oh, all the program names and their parameters would be listed, and oddities and picky details about these parameters would be noted. But much of the time their differences from a prior incarnation from whence the code was being reused were being noted; the prior code being long since lost or unavailable. If I want to know whether the function can work in two different modes? I’m out of luck, I’ll have to read through the code, or play around with it to see what it actually does. I would be impressed, if it didn’t mean I was the one wasting my time with things that should just be documented.

Or if the documentation wasn’t about how something was coded up, but about the concepts involved, this programmer would invariably pick obscure little details and go on forever about them. Big picture concepts? How the pieces fit together? Why the task is broken down into the constituent parts it is? Where additions or expansions are anticipated and how? Oh, no. None of that.

But I like to think that I’ve grown and profited from all of this. That my own documentation is the better for seeing how it shouldn’t be done. No pain, no gain and all of that. So I write lots of comments. I document. All intended to be useful and thorough. And sometimes, just sometimes, to be funny. So hopefully there will be a programmer someday slogging through my code. And not headdesking.

Expect’s a wonderful scripting language. Oh, it has its oddities, but I’ve been using it for over ten years now, and it’s proven useful in a variety of ways.

What is it? It’s a Unix based, fully featured scripting language based on Tcl that can take control of a user’s input and output and make decisions about what to do based on the results it gets. In other words, it’s designed to run programs that are meant to be run interactively by humans and put them under automated control.

Its uses aren’t unlimited: it’s definitely a command based scripting method — you’re not going to be controlling GUI windows with this. In checking around, there’s actually a few windows ports, but none of them really control windowed applications (one of them merely runs off cygwin!)

Where it shines is in automating tasks (especially of the system administration kind), and in testing (provided of course that no GUI is involved). I used dejagnu for years when I used to test network devices (routers, bridges, printservers, webcams, etc.

I can put together a script that checks for a new version of software, ftps it over, unpacks it, compiles it and then notifies me. I can then switch the current symlink to the latest version (or not) as I choose — all the grunt work is already done. I can do the same with any kind of repetitive task on unix.

I currently use it to automate the process of updating our services at work. Basically we offer a searchable database of texts, and we periodically upload new texts. The process involves quite a bit of “preprocess” — we go through the texts beforehand, find the words, and build several databases based on those words to help speed up the inquiries. We have searches based both on straight text search (slow, but can find substrings, etc) or a much faster word based search. The latter is accomplished through preprocessing. So first, we take the texts, and extract all the words. From this we can create indices based on unique instances of the words (678 occurrences of the word DOG in the database), or Btrees containing the complete location (filename and byte location) of every individual word. There are also databases recording the citations, others that help us create outlines for the material. You get the idea: lots of individual texts, many separate little programs to extract useful information, and all of it eventually welded together into a format usable by our website. Some of these tools are written in C. Others in perl, etc. We invoke the database command line to load things up. Files get copied over. Things get checked out, updated, committed from CVS.

Expect comes to the rescue here. It checks that we have the new set of texts, starts up each of the utility scripts in the correct order (checks that each completed without problems before advancing to the next). It also gives the user a number of options on how to do this: where the text files are located, whether we are on production or development machines, we can run parts of it (the prepatory work, versus actual loading of the data once preprocess complete). It handles safety stuff, such as dumping database schemas and cvs’ing them before dropping databases and resconstructing them or copying/tarballing files off before replacing them. All of the tools are under CVS control, and the script knows enough to check for any updates and recompile as needed before using the tools. If you completely forget to set the tools up, it knows how to cvs check out the entire thing from scratch. Depending on which machine we run this on, it can take up to 24 hours to complete.

I can start it up and walk away from it. It’s wonderful. My coworker still gets to do his stuff by hand cos he hasn’t taken the trouble to put together a script for his stuff…

Some useful links for more info:
http://en.wikipedia.org/wiki/Expect
http://expect.nist.gov/
Also, dejagnu: http://www.gnu.org/software/dejagnu/