Blog

[2010-06-20 22:48:43]: stak is looking forward to being unemployed.

There's a really comprehensive (and long) article in The New York Times about memories on the Internet. It highlights one of the things that's been stewing in the back of my mind for a few months now, ever since I decided to leave my job. The basic idea is that pretty much everything should have an end, and that people should know it.

The NYT article above talks about a lot of things relating to privacy on the Internet, and discusses the possibility of implementing expiration dates on data, after which the data would no longer be accessible (page 5 if you want to skip right to it). Given the way computers work now, and how easy it is to make digital copies, implementing this is practically impossible without a ground-up redesign of a lot of things. But it's still a good idea.

The specific expiration date that was relevant to me a couple of months ago was my last day at work. I wondered what it would be like if every employee, instead of having an open-ended full-time contract, had a limited-term contract which had to be renewed when it expired. This would allow companies to let go of employees who were under-performing, which increases the incentive for employees to perform at their best. Of course, it works both ways - the employer too must do their best to retain the good employees, since they would be free to go anywhere else once the contract expired. This would increase competition on both ends, presumably resulting in better conditions for everybody.

Note that all human societies that I'm aware of already use expiration dates in some important domains. As an example, consider how most elected officials are elected for a fixed term; without this sort of expiration date democracies wouldn't work at all. Human life itself has an expiration date (although it's not specifically known, and is more variable), without which evolution (both physical and societal) would not be possible. In both cases the presence of the expiry date allows for change and improvement at a much faster pace than would be otherwise possible.

With the concept of expiration, we also have to consider the concept of renewal. A lot of human-instituted expiration dates allow for renewal, where the expiration date is pushed back. For example, it's hard to imagine a fixed-term employment contract that doesn't allow for renewal. Why is it, then, that the president of the USA isn't allowed to serve for more than two terms?

I think that the more important the contract, the more important it is that the expiration of the contract NOT be allowed to renew. Death, after all, is not a renewable expiration date. In a sense, allowing it to renew undermines the expiration date in the first place. If you know that the expiration date can be pushed back, then you're going to behave accordingly; without that "the end is near" feeling, there's no impetus to complete whatever task needs completion, and no impetus to build a legacy worth remembering. Allowing the possibility of renewal also means that there has to be an entity that decides whether or not the renewal is justified. That introduces a whole raft of problems with criteria for renewal and subjective evaluation; enough problems, in fact, that entire religions have been created on this topic. Literally.

Another area in which I think expiration dates are becoming increasingly important are laws. Things like the USA PATRIOT Act came with built-in expiration dates on some of the sections, so that powers granted to the FBI and other agencies would only threaten civil liberties until the terrorist threat was taken care of. Unfortunately, some of those expiration dates were renewed and others ignored, resulting in the occasional abuse of power. And of course, everybody's heard of some of the really out-dated laws that are technically still valid. Not all of these problems would be solved by expiration dates more widely, but some certainly would.

There's a lot of other domains out there that could benefit from having expiration dates on contracts (using the term loosely), both of the renewable and the non-renewable kind. They just need to be selected and used judiciously, which won't happen unless people start thinking of them as a tool.

[ 5 Comments... ]

I was watching Barefoot Ted's talk at Google and one of the phrases he used got stuck in my mind (see video starting 50:25): can you handle being "other"? ("Other" in this context, if you don't want to watch the video, refers to doing something outside socially-acceptable norms.) I was thinking about what it meant and what it implied, and to me personally, there's a lot of richness to that one question.

Most people, by definition, aren't "other". They're "normal", they live according to society's rules, and they go about their lives without really questioning most of what they do or understanding why they do it. People who are "other" usually fall into two categories: one that actually gets a kick out of being labeled "other", and the other that has consciously chosen to become "other" because of some significant benefit it provides. To put in terms of the barefoot running example: there are people who will run barefoot just to freak out others by being "weird", and there are people (like Barefoot Ted) who run barefoot despite being labeled "weird" because it's actually good for you. It's important to distinguish between the two groups, and the stuff I'm talking about refers to people in the second group.

So why is being "other" important? Well recently I've been finding out more and more that things I take for granted because they're "normal" are really incredibly bad. In fact, a staggering amount of the lifestyle choices we make (in "developed" countries) at least are staggeringly bad for our health and/or happiness. We walk wrong. We eat wrong. We over-sanitize. We even poop wrong! It's no wonder that half a million to a million Americans die every year due to "lifestyle disease".

And the thing is, this didn't use to happen. A few hundred years ago "normal" was good. People had healthy lifestyles, for the most part. Sure they had shorter lives, but mostly because medicine back then wasn't as good as it is now. So how did this happen? There isn't really a single thing we can point to and say "Aha! That's where it all started going wrong!" If you've watched the Jamie Oliver TED talk, you can sort of see the progression though. He claims that people aren't taught at home or in school how to cook, so they end up making bad food choices. The question that follows is: why aren't they taught how to cook? Because their parents and teachers didn't think it was worth teaching. Why? Because they themselves didn't realize the benefits of proper nutrition over the convenience of fast food. Why? Well, until we started doing things wrong there was no reason to realize that we were doing things right to begin with.

So there it is: the human race was going along, doing things the same way that they had been for ages with proper nutritional meals. Then somebody came along and invented more convenient but less healthy food. We switched over to it, not realizing just how bad it was for us. I'm sure it was generally realized that it wasn't quite as healthy but the convenience factor overrode that by a wide margin. Then things start going real bad and everybody became obese. And now we scramble to try and fix it, after having learnt from the mistake and with newfound knowledge about the importance of nutrition. Fair enough. It seems like a pretty natural cycle, and I'm sure human history and evolution is full of cycles like this, where we make a mistake and then add to our store of knowledge about why it was wrong.

There's two things that come out of this, though. The first is that there is a lesson to be learnt from the "other" category. The very existence of the "other" category means that there's something they've discovered that is not general knowledge. By being open-minded and thinking about what they are saying, it's possible that you too can learn about the mistakes we've made as a species and can get back on the recovery curve faster. Once the "other" category spreads their knowledge back into the general population, they stop being "other". Being "other" is something like a transient state that exists only while something is wrong.

The second thing is that it's possible to realize the cycle is happening early on and try to prevent it. I know I keep harping on the same topics over and over, but I believe that driving is in the "omg this is awesome, let's all do it" phases of this cycle. It's been shown that driving is a significant cause of stress and promotes lack of exercise (not to mention that driving-related accidents are a leading cause of death). It's just all-around bad for you. People already know this, but... boy, it sure is convenient! Starting to sound familiar yet?

My dad came to visit recently, and one of the things he kept telling me was that I should get a car and how it's "essential for day-to-day life". He's no different than those parents in Huntington, West Virginia, who sacrificed health for convenience without even realizing it, and are now pushing the same choice onto their children. In this case, I have no problem being "other" by ignoring my dad and choosing more healthy forms of locomotion.

Your mission, should you choose to accept it, is to find a way to be "other" if you're not already. The point is to explore the choices we make every day, and to realize that we could already be wrong in a lot of ways that we don't realize. If you're completely stuck on ideas, this is a good one to start with.

[ 7 Comments... ]

Idealism is the responsibility of the young.

[ 0 Comments... ]

I was watching Jamie Oliver's TED talk (I like what he's trying to do, but after looking around his website it seems like his motives are more financial and narcissistic than altruistic) and started thinking about cancer. All the various forms of cancer are, at the root, caused by bad cell mutations. So really, in a way, cancer can be viewed as failed micro-evolution.

My theory-under-development goes as follows: when exposed to certain stressful environments (the stress can be chemical, physical, mental, whatever) the body tries to adapt to relieve the stress. It does this by becoming more prone to genetic mutations. The mutations, just like in evolution, are random, and the vast majority of them fail and result in diseases like cancer. In some cases the mutations are neutral (e.g. benign tumors) and in a relatively tiny set of people have successful gain successful adaptations to their stressful environments and live on.

The theory above is really far too generic to be of much practical value, but it does point to two possible cancer-prevention techniques. One is obviously to prevent the stress from occurring in the first place. I think most of the traditional cancer prevention techniques (e.g. these) fall into that category. The other is to find out how the body manages to make itself more prone to genetic mutations, and stop that from happening. I assume the body does this by secreting some sort of chemical when under stress (e.g. cortisol) which makes DNA more vulnerable to mutations. Another interesting point is that any such chemical is likely to be distributed throughout the body, which means that stress on one part of the body could cause cancer in a completely different part of the body.

Assuming the above is true, I'm not sure exactly which is the more "correct" solution to the problem. On the one hand removing the stress prevents the root cause and generally has wide-ranging benefits. On the other hand, some of the stress factors are just a result of how the world has changed over the last few millenia, and in those cases the more correct solution might be to leave the (possibly desirable) stress in place and instead force our bodies to adapt to it.

Note to reader: biology was one of my least-favorite subjects in school (too much memorization for my liking), and I know just enough biology to be dangerous.

[ 0 Comments... ]

Stolen from Slashdot: The Neuroscience of Screwing Up.

It's an excellent article based on observations of scientists working. I was particularly interested by the DLPFC as described in the article, since I can't see any benefit from having it. Evolutionarily speaking, that should have been eliminated a while ago. And I totally agree with the article that explaining your thoughts and/or problems to somebody else will often directly help you find a solution. I've often started a blog entry about some thought I had, only to realize halfway through that it didn't really make as much sense as I thought it did. Those blog entries never see the light of day, but they do help me filter out some of my misconceptions before they become too deeply ingrained. Add that to the list of reasons why blogging is useful (provided you actually blog about thoughts and ideas, rather than just journaling what you did).

[ 2 Comments... ]

I was reading this article about MinWin and what they're trying to accomplish. I hadn't really understood (or cared) what it was before, but after reading that article I gotta say, I'm really glad that they're doing it. I can feel their pain in trying to untangle large piles of old code, because I've been doing much the same thing at work for the past little while.

There's code that's been lying around for around 8-9 years, predating anybody on my team. There are some fundamental differences between code like this and "new" code. For one, there's nobody around who still understands what the code does, and obviously any documentation is so out of date that it's only purpose is to mislead you. The only way to understand it is by examining the code itself - you have to read it, poke at it, and break it.

I believe that in pretty much any real-world production system, built under real-world constraints, there will be some code like this. In order to maintain it, or even to rewrite it, being able to read and understand code is a fundamental skill. I think of this as another argument against writing documentation. If developers need to acquire the skill of reading code anyway, then you might as well use that skill on new code as well. This makes the documentation redundant.

Of course, that's not all. Usually when reading new code, variable names and classes do have some relationship to the concepts they represent. The older the code is, the weaker that relationship becomes, because the concepts get shifted and skewed whereas the names do not. As far as I can tell the main reason this happens is because it's just a chore to rename things, particularly in systems where the code has been branched into different versions. Integrating fixes after you've renamed things (particularly in Java, where you have to rename the file if you rename the class) is a major pain with no tangible benefit.

I feel the blame here lies mostly on revision control systems. Renaming a file in, say, Bazaar is trivial compared to the same operation in Perforce. An even better revision control system (which incidentally is also my solution to the subjective readability problem) would store the parsed syntax tree of the code rather than a flat text file so that operations like variable renaming could be tracked as a single change and integrated into branches trivially. Such an RCS would also have a long list of other advantages, but I'm not going to get into that until I start writing one :)

Another thing mentioned in the MinWin article is how "countless spaghetti strands extend outwards from the core of Windows to the layers higher up in Windows" - this is basically the programmer's version of dependency hell. When you have dependencies running amok between different parts of the code, everything gets really bad really fast. It becomes easy to end up with circular dependencies - to solve that you either end up compiling both pieces of code as a unit so the compiler can deal with it, or changing one of the dependencies to be some sort of runtime/reflection thing, which makes the code an order of magnitude harder to follow. Code like this is also (by definition) not very modular, and so is hard to unit-test.

When we were writing Mango, one of the design principles we enforced was that even though all the code was compiled together as a unit for production use, the packages in the code were arranged in a DAG. This allowed us to build -- and more importantly, test -- subsets of the rendering engine with each layer adding more functionality to the previous subset. The MinWin team seems to be realizing similar benefits in being able to build standalone subsets of windows for different purposes. In my subjective opinion, of all the design decisions we made, this was probably the single most useful one. Without it, the code would have collapsed in on itself and become an unmaintainable mess within a year, given the rate at which we were churning out code.

Enforcing that design decision from the start was key, though. As I'm discovering with my current refactoring efforts, it is extremely difficult to handle code that was developed without that sort of modularity. Coercing the code into a more elegant design requires several passes of refactoring and lots of time. I'm just thankful I have a smaller codebase to deal with than the tar pit that is Windows.

[ 0 Comments... ]

Problem: social networking cloud services (e.g. Facebook) have to rely on advertising for revenue. This obviously annoys users and isn't very reliable.

Problem: people spend a lot of time at work on these social networking sites, in some cases prompting their employers to block access to said sites. This results in lowered employee morale (although arguably better productivity).

Problem: larger companies attempt to replicate the service in-house. Invariably, communication and collaboration software developed specifically for enterprises is crap. In addition, employees need to maintain two different social network accounts with overlapping functions.

Solution: the Enterprise Enhancement Proxy (EEP). A proxy server that is purchased by the enterprise from the cloud operator and placed inside the enterprise firewall. This proxy serves a dual purpose.

(1) It serves as a proxy for all communication to the public cloud, allowing the enterprise network administrators to filter or block certain types of traffic at a more granular level than currently possible (e.g. you probably do not need to publish facebook videos while at work, although status updates might be acceptable).

(2) It interfaces with an enterprise-local database to seamlessly (or maybe seamfully) integrate public and enterprise-specific social data. The resulting view would include social data from both public and enterprise networks, and any interactions with the enterprise data would remain within the enterprise database itself.

So, for example, let's say Twitter develops an EEP and it is deployed at some corporation. When employees on the corporate network go to see their Twitter feed on twitter.com (or any app that accesses the Twitter API), the EEP will intercept the request and mix in tweets from their co-workers into the resulting view. Any replies from an employee to a co-worker will get intercepted by the EEP and get saved to the enterprise database, and will not be visible on the public internet.

This solution allows enterprises to retain full control over their intellectual property while also taking full advantage of the services provided by public social networks. It also provides the social networks with a more reliable revenue stream. The users benefit too by not having to maintain separate social network accounts for public and corporate use. Everybody's a winner!

[ 8 Comments... ]

If anybody else would like to download the contents of their Gmailbox into nice little RFC822 message files, here's a quick-and-dirty (read: no error detection or handling) java program to do it. Requires java 1.6 for the console stuff, but can be ported to 1.5 or 1.4 in a pinch.

import java.util.*;
import java.io.*;
import java.net.*;
import javax.net.*;
import javax.net.ssl.*;

public class GmailDownloader {
    public static void main( String[] args ) throws Exception {
        int start = 1;
        int end = -1;
        if (args.length > 0) {
            start = Integer.parseInt( args[0] );
            if (args.length > 1) {
                end = Integer.parseInt( args[1] );
            }
        }

        Console console = System.console();
        String username = console.readLine( "Enter username: " );
        String password = new String( console.readPassword( "Enter password: " ) );

        SocketFactory sf = SSLSocketFactory.getDefault();
        Socket socket = sf.createSocket( "imap.gmail.com", 993 );
        InputStream in = socket.getInputStream();
        BufferedReader br = new BufferedReader( new InputStreamReader( in ) );

        OutputStream out = socket.getOutputStream();
        PrintWriter pw = new PrintWriter( out );
        br.readLine();

        pw.print( "A LOGIN " + username + " " + password + "\r\n" );
        pw.flush();
        br.readLine();

        pw.print( "B SELECT \"[Gmail]/All Mail\"\r\n" );
        pw.flush();
        for (int i = 0; i < 3; i++) br.readLine();
        String numMsgs = br.readLine();
        for (int i = 0; i < 3; i++) br.readLine();
        if (end < 0) {
            StringTokenizer st = new StringTokenizer( numMsgs );
            st.nextToken();
            end = Integer.parseInt( st.nextToken() );
            System.out.println( "Found " + end + " messages" );
        }

        System.out.println( "Downloading messages from [" + start + "] to [" + end + "]" );
        for (int i = start; i <= end; i++) {
            System.out.println( "Downloading message " + i );
            pw.print( "ZZ FETCH " + i + " RFC822\r\n" );
            pw.flush();
            br.readLine();
            String fn = i + ".msg";
            while (fn.length() < 12) {
                fn = "0" + fn;
            }
            PrintWriter file = new PrintWriter( new File( fn ) );
            outer: while (true) {
                String s = br.readLine();
                while (s.endsWith( ")" )) {
                    String t = br.readLine();
                    if (t.startsWith( "ZZ OK" )) {
                        if (s.length() > 1) {
                            file.println( s.substring( 0, s.length() - 1 ) );
                        }
                        break outer;
                    }
                    file.println( s );
                    s = t;
                }
                file.println( s );
            }
            file.close();
        }

        pw.print( "C LOGOUT\r\n" );
        pw.flush();

        in.close();
        out.close();
        socket.close();
    }
}

The above code is in the public domain, so feel free to do whatever with it. To use:

javac GmailDownloader.java
java GmailDownloader

If it dies partway through and you want to resume, just add the message number you want to start at as a parameter:

java GmailDownloader 42

[ 3 Comments... ]

For anybody else out there who's stuck on trying to get OpenBSD booting on a macppc machine (mine is a 12" Powerbook specifically), the magic that is missing from the OpenBSD install manual/instructions is as follows:

After installing OpenBSD, when you power up your system, hold alt+command+o+f to boot into Open Firmware. Then, at the prompt, type "boot hd:,ofwboot bsd" and hit enter. If this successfully results in OpenBSD booting up, you can make this the default behavior by booting back into Open Firmware and typing "setenv boot-device hd:,ofwboot" and "setenv boot-file bsd" at the prompt.

Note that the above worked for me when I installed OpenBSD to take up the whole disk; I don't know what the procedure is if you're dual-booting with Mac OS or any other OS.

[ 0 Comments... ]

Schneier's post on self-enforcing protocols reminded me of a similar scheme I thought of a while ago for negotiating the price of something when there's a single buyer and seller. In order to get the best price for both parties, the buyer should write down the maximum price he is willing to pay. The seller should write down the minimum price he is willing to accept. If the buyer's number is greater than or equal to the seller's number, then the price is the average of the two numbers. If the buyer's number is smaller, then there's no deal. The key is that this process happens only once, so neither side has a chance to cheat and adjust their number later. If they want the deal to actually happen then they should be honest about their max/min bids, and that automatically results in the fairest price for both parties.

[ 1 Comment... ]