Tech

Playing around a bit with ssh tunneling right now. When I create a kind of VPN concentrator with a few tun interfaces: is there any reason why I can't just assign the same IP on all these tun interfaces? A quick test shows this set up working nicely, with ifconfig tunX localip pointopoint remoteip (the localip part being the same) setting up the routes to chose the right tun device for all remote IP addresses, and ping worked just fine for me. Firewall rules will always have the remote IP and/or the interface name to decide when a packet applies.

Obviously setting up a listening socket at only one of these interfaces is not so trivial now since I can't just listen to the IP, but that's a restriction I'm happy with. Anything else I'm not thinking of right now?

I've had alias net-pf-10 off and alias ipv6 off in my configuration for ages, but with recent kernels, IPv6 is apparently compiled in, so my eth0 now has a (link local) IPv6 address.

... which means that I now had Java apps suddenly fail to connect to the database. Since stuff like sqlplus (Bah!) works, I didn't immediately think about a network problem and started testing various JVM versions and hunting missing libraries etc. Only, it turned out that running java with -Djava.net.preferIPv4Stack=true is the magic bullet.

Is there a proper fix? I know IPv6 is (supposed to be) the nice and shiny future, but currently I just don't have any IPv6 connection so it just creates problems for me right now. (Ok, let's add here that I dislike Java anyway because its philosophy, or the philosophy of most software I have to deal with, is to define its own world instead of integrating nicely with the host platform. If you look at it from another direction you may think I should have titled this “I suck”)

Wow. I just stumbled over the SIL Fonts, which I haven't heard of before. Looks like an absolutely great project. Not only are these fornts released under an open license (I haven't read it myself, but many of these fonts are in Debian...), but above all I really liked the selection of supported systems with icons by each entry: The Old Windows 3.1 Windows flag stands for “tested on Windows”, the very old rainbow colored apple for “tested on Mac”, our official (and current :-) familiar swirl not only for tested on Debian, but for “Debian package available”.

I mean: while it's not uncommon today to see some kind of Linux support in software (or other downloads), explicit Debian support is not that widespread. And seeing this side by side with the two very outdated logos for the two other mainstream OSs (and Ubunut not even being mentioned on this page) is ... what, exactly? Wishful thinking? An ironic comment on how the (conputing) landscape should look like? Made me smile, in any case.

I know, preaching to the choir and all that. But this can't be said often enough (and perhaps the odd non-Debian person stumbles upon this...): Here's another example of how amazingly fast free tech support works: The newest Linux kernel (Debian package version: 2.6.32-1) wouldn't boot on my QNAP TS-419P NAS (see also my earlier posting about the device). Now, since I don't have a serial console cable, I can't really help a lot to debug this (and I am extremely happy that the people at QNAP have thought about recovery: it's trivial to just flash a working kernel or firmware image via DHCP and TFTP), bu still Martin Michlmayr immediately took the time to help me and soon could reproduce it on one of his own machines. The problem has been identified, I got a fixed kernel, and a patch is on his way to the kernel maintainers. All within literally just a few days.

Second part (and biggest in terms of space) is my new Lenovo desktop computer with a nice NEC 26" screen. The screen is quite a bit better than my bulky 19" CRT, but the story of buying the desktop was more involved.

I've read the widely reported Linux Foundation announcement about new membership benefits and promptly fell for it. Thumbs down for LF, because they absolutely failed to mention that the discounts on HP, Dell and Lenovo products are only available within the U.S. [insert here: cheap joke about U.S. Americans not knowing that the rest of the world even exists.]

Thumbs up for Lenovo, though: after a few emails with both the LF and Lenovo, I got a 40% discount on their original price for my system.

Now obviously big brand computers like these are still overpriced (or less powerful at the same price) when compared with a white box. And I almost went with a barebone, CPU, etc. because I quite like putting hardware together. On the other hand, I haven't really kept up with CPU socket types etc., and since I absolutely wanted an Intel GPU, the 36 months on site support won out. And when the new computer went beep beep beep (and dark) on the third or fourth boot, I was quite happy that I didn't go the white box route after all...

I won't bore you with installation details (I'll be getting back CHF 45 for my unused copy of Vista, of course), suffice to say that today's dual core 64 bit 3GHz / 4GB RAM machines are a bit faster than the 32 bit ca 1 GHz / 1GB RAM with mostly unaccelerated graphics from five years ago...

After I had been living with mostly the same electronic toys over the last few years (ok, I got the netbook earlier this year, but that's about it... and it might even count as a serious work tool given how much I came to use it) all the toys I've agonized over buying suddenly appeared on the doorstep within these two weeks...

First part was the QNAP TS-419P which I bought after reading about QNAP on Martin Michlmayr's blog. The decision was quick when I realized that at least part of the Debian related information was not hidden in some obscure web forums but was there on the QNAP wiki itself, indicating quite some level of support from the manufacturer. Beyond that, tbm does an awsome job supporting QNAP users like myself through his excellent QNAP / Kirkwood pages and also, since Debian installer support for the 419 is not quite finished, by email. Kudos to him!

So, once I get past some remaining obstacles (for some reason, the initrd refused to mount a raid1 root device when I tried it at first; working on that, but it's not that easy to debug without console access...) our old storage / Internet router (very old! Pentium II 350MHz with 128M RAM and its probably third internal disk, which became ridicolous to use when I bought a (big) external USB disk and realized it only has USB 1...) can be retired. Should be good for the environment, too, I should hope, in terms of power consumption even considering that I now run 3 more disks.

Just read Mark Kretschmann about application naming.

Another consideration is: will the name show up in Google? Some projects have been extremely bad at this, the prime example I'm thinking of is the KVM virtualisation module in Linux. I'm treating KVM as the name here, since the term “Kernel-based Virtual Machine” is not even a name, really... (ok, it has become popular enough so that linux-kvm.org now shows up at the top.)

I was just seeing the headlines about Google's “SPDY” technology which, as far as I understand right now (haven't read the LWN article yet) is trying to accelerate HTTP by doing fancy things within the TCP stream.

I'm just thinking: why not do HTTP over UDP instead, like DNS does? This would cut down latency for small requests a lot, and for big transfers, the added latency woulnd't matter that much. How could that work?

• For starters: either the browser always tries UDP first and falls back to TCP, or the server could advertise UDP capability in its HTTP headers. Either way, browser would need to have a small cache on where UDP can be used. Given that all browsers already carry lots of databases (content cache, cookies, pop-up policies, form completion, ...) I don't hesitate to propose this additional one...
• If it is established that a server for a domain can handle UDP requests, lots of heuristics could be used to determine wether the browser should send UDP or TCP requests; the end result would always be that the server is free to either respond with the content in an UDP packet if it fits, or decline an UDP answer with a “please use TCP” 3xx response code.

I figure that with today's JavaScript enabled, Ajaxy web, many requests (and responses, of course) would indeed be small and would fit in an UDP packet (especially since I'd say content should be compressed in any case without special negotiation.) OTOH, encryption would be hard (or, at least in a first protocol revision, not supported) and since persistent TCP connections are available, maybe this is just unnneccessary. I don't know. But I'm not sure if persistent TCP connections are actually that widely in use on web application where a user might click on a button every few minutes. How long do browsers and servers keep the TCP connection open?

Yes, this is a dear lazyweb...

I'm more and more confused about IPSec on Linux. Has anybody done a more or less recent summary about IPSec and Linux? All I find is quite old and probably doesn't apply anymore... About the only thing I'm sure about is that FreeS/WAN is officially dead. But otherwise? Openswan vs. strongSWAN vs. KAME (or is that ipsec-tools?) And what about the kernel side? Are they now all using the same code or are there still patches?

It would probably best if the hypothetical knowledgeable person would not comment here but insert a relevant notice at the top of http://wiki.debian.org/IPsec, to avoid having yet another obscure page (mine) show up in Google...

Yes, we're all doomed. Especially, Debian is doomed. We're relying too much on public key cryptography, which is based mostly on the factorization problem (getting the two big prime numbers that essentially make up a public key is hard.)

And now they [Link to Bruce Schneier, where I've got the news from] have gone ahead and have built a working quantum computer, which can solve the factorization problem very quickly. So, will we have to re-check all Debian software that was signed with an RSA key in the last few months?

The huge number that the quantum has successfully cracked was:

15

Email on a desktop Unix system is apparently a hard topic. At least, so far nobody seems to have solved it to my satisfaction... That's where the Pony Express comes in: a sensible /usr/bin/sendmail implementation for desktop sysstems:

• No queue, no running processes.
• pass users' email to a mail hub
• support per user outgoing mail hub and SMTP authentication + TLS.
• still allow local mail delivery, so stuff like cron will deliver to local mailboxes.

Mercurial repository is available; Pony Express is written in Python 3. I'll not upload a Debian package right now since as far as I've seen Python 3 is not supported by the usual Python helper scripts. Status: Lots of missing features and probably lots of bugs, but I use it from kmail to deliver mail, and the commandline "mail" also works. But that's the extent of the testing it has received so far, and in addition it's also one of the first things I've ever done in Python, so consider yourself warned. Still, feedback and patches welcome.

Update: Daniel, no, I can't. I missed msmtp when I looked for a solution after I saw that nullmailer and ssmtp wouldn't do what I wanted.

Update: cate / simon: my “apt-cache search” skills obviously suck, I didn't find esmtp either. Oh, well... I wanted to learn Python anyway, and since Pony Express is basically a frontend to Python's smtp library, it was not that much work. lamson sounds interesting for other projects, I'll have to look at it. From the description it's not what I want here, though. Marius: kmail does its own queueing, so I usually use the send later option and send non-local email from the commandline only while connected. I'll have to think about a non-daemon, non-cron queue style that suits my need (event-triggered from if-up.d perhaps), but as soon as I start to accept mail that I can't immediately get out of the system complexity goes up quite a bit. Developing PX into a full MTA would still be a fun project and I've got a few other ideas that I could incorporate. Not tonight, though ;-)

Ok, I admit I did that to get you to read the article...

I'm just a bit saddened that Microsoft Exchange compatibility seems to be the holy grail of Open Source (Free Software if you like) PIM application developers. Right now I've read Gary Greene's announcement that he'll bring KDE PIM closer to Exchange, but other PIM clients (Evolution, IIRC) are also working hard at being good frontends to Exchange servers. Meanwhile, I've run into problems with various Open Source groupware servers (strictly speaking Open Source, you'll probably remember me ranting about this before) when trying to use KDE PIM.

So, the the way into the future seems to be to run KDE PIM with the soon to be completed ;-) MS Exchange connector, running an almost-Open Source groupware server with a commercial Outlook Connector plug-in. (Update: Clarification: running a groupware server with an outlook connectivity plug-in. I think Zimbra offers something like this, for example.)

(To give you a bit background data: We run Zimbra at a client, which has a nice web frontend and runs very well with its own fat client. Using KDE PIM is not really an option, too many features don't work. Open-Xchange was what we had there before Zimbra; KDE PIM integration is not really possible either, they don't have a fat client, and while the web frontend is nice, they had a poor track record with reacting to issues (and we did have the paid for version with 150 users, and at least one integrator claimed that with 150 users we were big enough that they would react immediately to our complaints...) I had a very quick look at eGroupWare, but that was right around the time when Tine 2.0 forked away from it, so we weren't sure where the community was going. I didn't look closely at OpenGroupware.org/SOGo; from what I remember they dropped out of the evaluation early because of missing features.) Kolab should, as far as I know, have the desired integration with KDE PIM, but I'm not sure what the status is on that front, and when I last looked, there was basically no web frontend at all, and the KDE PIM integration did at least at some time in the past require its own version of the KDE appliactions.)

I've observed this several times: for people (sysadmins, not end users) coming from Klickibunti systems, using the Linux commandline follows a “reverse 80-20” rule. The first 20% of skills are the hardest to learn, after that it's all gravy. I'm trying to identify what fundamental concepts are usually missed by people who learn mostly by copying commands from recipes they copy from Google search results. (If you comment: my blog sadly eats formatting, so if you write more than a few lines it may be better to write to your blog and trackback-ping me.)

Paths and the Working directory: Files without paths are created/looked for in the current working directory, the same applies to the argument of the cd command. Paths beginning with '/' are absolute, and this is handy because it is not necessary to repeatedly cd .. before descending to some other path. (And yes, directory names can be concatenated to paths instead of using repeated cd commands with a single directory name.)

Stdin/out/err and shell redirection: The typical question here is whether | less works with this-and-that command, too. Admittedly, stderr and stdout are not used consistently, and commands like ssh who use the terminal instead of stdin (and for good reason) don't make this lesson easier.

Shell commandline expansion: Shell globs and variables etc. are expanded by the shell before calling the command. Accordingly, the rules are the same everywhere, and this nice trick of using [A-Z]* when copying files works for deleting files with rm, too. This is tightly linked with the first concept shown here, and lack of understanding here directly leads to spaces in filenames.

Regular expressions are useful. They are not just something I insist you learn, but they actually help you do your work. They're everywhere. If you understand grep and understand about redirection, the world is your mollusc. Somehow related is that I can not understand how somebody can refuse to learn a decent text editor. One that runs in the text console, so you can use it in ssh, and one with built-in search and regex based replace. It doesn't need to be vim (although the fact that a vi style editor is available everywhere is an advantage.)

. is not in $PATH: This is, of course, the whole chain of understanding what '.' is to what shell variables are, at least in principle, to what $PATH does to executing a command in a directory, not by going there and then running ./command but by specifying the path to the command to the final ah-ha that using ./command actually is specifying the path to the command (and not just “it works that way”.)

No news is good news: Typical sign is running cp twice (ah, we've discovered cmdline history) to make sure it works or running rm twice to make sure that the fike is really gone.

And then there is the secret rule: Sometimes you don't want to know. The Unix world is full of arcane details where asking why doesn't get you anywhere (not technically, at least. Learning the history of Unix and Unix-like systems is fun, interesting and may contribute to a deeper unerstanding in the end.) Most programs use getopt style argument processing, but dd is just different. The directory /usr is not a directory where users store their data. There are sed style regex (regexes?) and there are pcre. Firefox is called Iceweasel in Debian. There are vi style console keys (less, view, vi, and yes, the man viewer is less) and emacs style console keys (default bash settings, the info viewer.)

Thinking that this might be worth hacking on tail (because I felt the functionality would be better kept there than in head), I started to have a look at its source. Lo and behold! tail -n +N already does exactly this...

(Obviously a followup to Zack's comment on my previous posting.)

A script I keep reinventing again and again: installing those binary packages from a self-compiled package where the other (presumably older) version is already installed on the system. I don't have time to make this generic right now, but still...

for i in *deb; do 
    echo ${i%%_*};
done | xargs dpkg -l 2>/dev/null \
    | buthead 5 \
    | cut -f 3 -d \  \
    | while read p; do
         echo  ${p}_4.2.3-1vbi_i386.deb;
      done \
    | xargs sudo dpkg -E -i

Update: I don't know how I managed to see -E in dpkg's manpage but miss -O which is right above it. Thanks, Guillem. Josh: Either my english is just not good enough, or the manpage of debi is from a slightly different reality. I just can't make out what, exactly, the tool tries to do. (Ok, since dpkg -iOE does exactly what I want anyway I didn't try very hard either.)

Update: RTFM.