LISP

Although I can imagine needing some time to get used to functional programming, I think I would enjoy it, and I think that it would make me a much better engineer. Many of the idioms seem so powerful (map, filter, reduce, etc.)

So, I need to learn a functional language. But which one? Common LISP, OCaml, Haskell, Scheme,...? Suggestions?

Frameworks

I was reading something the other day and it occurred to me that Java developers heap framework upon framework, often to abstract away the details of an underyling framework. Struts, Hibernate, Spring, and a bunch of others.

Compare this morass with the (relative) simplicity of PHP, Ruby on Rails, Django or TurboGears... (Yes, I know that J2EE may be more scalable...but how many sites truly need such scalability?)

Java: The Kingdom of Nouns

This is absolutely brilliant.

Great Quote

"XML is some kind of giant accident, imo. like, there was some kind of unfortunate coincidence whereby a whole bunch of managers were all standing in magazine shops, and they all saw the letters XML on the front of a computer magazine in bright red letters, and they all went back to their companies telling everyone they should be using XML"

Java and Python: A Small Comparison

I have a file called "numbers.txt" that lists the numbers 1-10 on separate lines. I want to make a small program that reads all of the lines and prints the numbers to stdout in the following format:
1 2 3 4 5 6 7 8 9 10

Here's the somewhat minimalistic Java code to do this:

import java.io.BufferedReader;
import java.io.FileReader;
import java.io.IOException;

public class Numbers
{
 public static void main(String[] args) 
 {
  try {
   BufferedReader br = new BufferedReader(new FileReader("numbers.txt"));
   StringBuilder sb = new StringBuilder();
   String str;
   while( (str = br.readLine()) != null) {
    sb.append(str.trim() + " ");
   }
   sb = sb.delete(sb.length() - 1, sb.length());
   System.out.println(sb.toString());
   br.close();
  } catch(IOException ioe) {
   ioe.printStackTrace();
  } 
 }
}

Now compare that code with Python code that does the exact same thing:

if __name__ == "__main__":
 try:
  sock = open("numbers.txt", "r")
  print " ".join([l.strip() for l in sock.readlines()])
  sock.close()
 except:
  print "Error reading from numbers.txt."

Several things jump out at me here:
1. The Python code is much more expressive.
2. The Python code is much closer to the way I think.
3. The Python code doesn't make me deal with nearly as many mundane, niggling details. For example, Python doesn't force me into an object-oriented paradigm (even though I could go there if I wanted to).
4. The Python code to me 30 seconds to write. I needed roughly 3-4 minutes to produce the equivalent Java code, because I had to look up appropriate method calls in StringBuilder and BufferedReader. In addition, I had one compilation error with javac (I had originally named my file "numbers.java" instead of "Numbers.java").

Obviously, the example described here is somewhat contrived; nevertheless, it illustrates some of the reasons why I like programming in Python and why I loathe programming in Java.

Quote of the Day

Sayeth Aaron Krowne: "Saying you can have only economic conservativism and social liberalism, OR economic liberalism and social conservativism, is about the most ridiculous false dilemma, political trick that's ever been played on 300 million people."

How to Spin the News with Statistics

There's a link on cnn.com called "Watch: Scary numbers in America." (The linked URL is actually Javascript; hence, it won't do any good for me to link to it. Also, if you want to view the story for yourself, note that it requires a Windows Media Player plugin.)

I'll save everyone the trouble of watching it by posting a summary:

• 1. 37 million Americans live in poverty.


• 2. U.S. high school students rank 28th in the world in mathematics.


• 3. U.S. high school students rank 22nd in the world in science.


• 4. 50% of Hispanic and black teenagers will not graduate high school.


• 5. Last year, 1.5 million babies were born out of wedlock in the U.S.


• 6. Washington, D.C., has a higher infant mortality rate that some cities in India


• 7. U.S. is no longer the most college-educated nation.

Let's take a look at these one by one.

1. There is no doubt that there are a lot of poor people in America. Unfortunately, there probably always will be.

However, there is much that CNN isn't telling us. For example, consider the following excerpt from a New York Times piece by Nicholas Eberstadt:

The profound flaws in our officially calculated poverty rate are revealed by its very intimation that the poverty situation in America was “better” in 1974 than it is today. Those of us of a certain age remember the year 1974 - in all its recession-plagued, “stagflation"-burdened glory. But even the most basic facts bearing on poverty alleviation confute the proposition that material circumstances in America are harsher for the vulnerable today than three decades ago. Per capita income adjusted for inflation is over 60 percent higher today than in 1974. The unemployment rate is lower, and the percentage of adults with paying jobs is distinctly higher. Thirty years ago, the proportion of adults without a high school diploma was more than twice as high as today (39 percent versus 16 percent). And antipoverty spending is vastly higher today than in 1974, even after inflation adjustments.

In the face of such evidence, what do you call an indicator that stubbornly insists that the percentage of Americans below a fixed poverty threshold has increased? How about “a broken compass?”

The soundings from the poverty rate are further belied by information on actual living standards for low-income Americans. In 1972-73, for example, just 42 percent of the bottom fifth of American households owned a car; in 2003, almost three-quarters of “poverty households” had one. By 2001, only 6 percent of “poverty households” lived in “crowded” homes (more than one person per room) - down from 26 percent in 1970. By 2003, the fraction of poverty households with central air-conditioning (45 percent) was much higher than the 1980 level for the non-poor (29 percent).

Besides these living trends, there are what we might call the “dying trends": that is to say, America’s health and mortality patterns. All strata of America - including the disadvantaged - are markedly healthier today than three decades ago. Though the officially calculated poverty rate for children was higher in 2004 than 1974 (17.8 percent versus 15.4 percent), the infant mortality rate - that most telling measure of wellbeing - fell by almost three-fifths over those same years, to 6.7 per 1,000 births from 16.7 per 1,000.

Hmmm...so Americans are poorer than in 1974, according to CNN and the government, yet they are better off and have a higher standard of living. What could possibly account for this? Could it possibly be that the way in which poverty is measured is inherently flawed?

The poverty rate is out of step with all these other readings about deprivation in modern America because it was designed to measure the wrong thing. The poverty rate has always been derived from reported household income. (Exigency played a role here: at the start of the war on poverty 40 years ago, those income numbers were already available from the Census Bureau.) But a better gauge of a household’s material deprivation is not what it earns, but what it spends. When we look at spending patterns, we immediately see a huge discrepancy between reported incomes and reported expenditures for low-income Americans.

In the Labor Department’s latest Consumer Expenditure Survey (2003), the average reported income for the bottom fifth of households was $8,201, while reported outlays came to $18,492 - well over twice that amount. Over the past generation, that discrepancy widened significantly: back in the early 1970’s, the poorest fifth’s reported spending exceeded income by 40 percent.

Of course the "poverty rate" is an inherently flawed "statistic." I cringe every time I hear it used, because it doesn't tell us much of anything. There are two incredibly gaping flaws with this measurement:

• As the quote above mentions, the poverty rate is measured according to household income, not spending, net worth, or wealth. Therefore, a millionaire could technically "live in poverty" as far as the government is concerned.


• The poverty rate is not adjusted for region, as far as I know. Since $16,000 per year will go a lot further in northern Mississippi than in the greater Boston area, for example, it seems as though one would want to use a calculation technique that takes these differences into account. We should cut the Census Bureau and CNN some slack though, for flawed statistical measurements are much easier to make. They're also much more useful, provided that they give the impression that CNN wants its viewers to have

Anyway, I'm just wondering why CNN or any other 24-hour news outlet never raises these sorts of points in their broadcasts.

2. Can anyone tell me what we are measuring when we report that U.S. students are 28th in math?

Here's the problem: In Germany (and other European countries that are doubtlessly ahead of the U.S.), students take a big, important test around eighth grade or so. If they perform well on that test, they enroll in a college preparatory Gymnasium (high school); otherwise, they enroll in a trade school. Now, recall, for the moment, that in U.S. public schools, everyone takes standardized tests and everyone takes the same basic curriculum. So, is it the case that we are comparing the upper echelon of students in countries like Germany with everyone in the U.S. in order to rank countries based on standardized test scores? I honestly don't know, but I wonder why CNN never clarifies points like these.

Statistical sampling aside, I'll be among the first to admit that the state of mathematics education in this country is deplorable. I remember working in food service back in high school when one of my co-workers couldn't make change for an $8.25 lunch when she was handed a $20 bill. Unfortunately, the lack of basic mathematics skills is probably typical of today's high school student. The reason for this, in my opinion, is that we, as a society, no longer really value education. We're more concerned that Brittney Spears had a kid. We care more about Tom Cruise's religious beliefs. School administrators are more concerned with teaching to a test so that they can make a good impression with state and federal regulators as opposed to teaching kids how to think. Make no mistake about it: Math is not, and never will be, about regurgitating algebraic formulae; it's about analyzing problems and applying a standard corpus of techniques to them. It's also about discovering new techniques to solve problems.

So there you go...if you can change the culture of this society and the accountability mechanisms in the public school system, math skills will improve.

3. See #2.

4. This is due to a mixture of poverty and ambient cultures that do not value education. No amount of educational funding increases will change this statistic significantly.

My question: Where are Jesse Jackson and Al Sharpton on this issue? This is one issue on which they might be able to make a difference. I suppose education breeds independence, though, and these two men thrive on cultures of ignorance and dependence.

5. The degradation of the family is perhaps our biggest cultural problem. Children need two good parents for emotional and economic stability. For me, this statistic is the most concering and dishearting of all. It is inextricably linked to #1, a point which CNN didn't bother to mention.

6. Thank you, CNN, for THE MOST USELESS statistic EVER. Comparing the most dysfunctional, screwed-up city in America (in terms of drugs, crime, and poverty) with some cities in India that you don't even bother to name is even a new low for you! So, congratulations!

Question: How many of these infant mortalities were born to crack- and heroin-addicted mothers? Could that account for the higher rate? I wonder why CNN didn't address this.

7. WHO CARES? Seriously, why is this important? Why does the media seem to think that a college education is the be-all and end-all? College is not right for everyone, and it is often unnecessary.

Does anyone seriously think that farmers, ranchers, auto mechanics, carpenters, electricians, and plumbers need a college education? These are all skilled labor positions that are vital to the survival of our economy, and in a lot of respects, these people are much more educated than some 22 year-old English major fresh out of college. Yet no news media outlet ever points this out. No news media outlet ever celebrates the skilled labor positions that keep our economy rolling along. They simply lament that the percentage of people going to college is lower than in other countries. Make no mistake about it, this is a non-statistic. There's no substance here; CNN is merely blowing smoke.

War on Drugs

Make no mistake about it, the "War on Drugs" is the single largest source of civil rights violations in the United States today, yet politicians and DEA agents are hardly ever held responsible for these abuses that are ENTIRELY THEIR FAULT.

Unfortunately, the following scenario is all too typical in the history of the War on Drugs:

"Rudy was a father of five who was passing by a house targeted by narcotics officers attempting to serve a parole violation warrant and the police mistakenly thought he was the one they were there to arrest. They chased Cardenas, and he fled, apparently afraid of them (they were not uniformed). Cardenas was shot multiple times in the back."

DrugWarRant.com chronicles several examples like this one and is worth a read.