January 2007 Archives

Two weeks ago I wrote about a presentation by Scott Page on diversity and the wisdom of crowds. One of the examples he used was the greater effectiveness of polling the studio audience versus calling a friend for an answer to a question. I suggested that a big part of this Millionaire problem is the difficulty we all have of seeing into networks, whether they belong to others or ourselves.

Over the summer I created an account on LinkedIn and played around with it for a few weeks. The initial excitement of adding people I knew from school or work soon faded and I haven't been back to the site for a few weeks. I just wish it had a few more interesting features:

• Make it into a distributed polling application instead of focusing on introducing people to one another. And it looks like they've added that feature in the “Answers” area. I guess this either shows that I'm thinking a few days ahead of the curve or that like minds run into the same problems. The “Answers” section needs an RSS feed and quick.
• Now take it further and enable it to allow people to form teams for projects of any kind, from starting up a company to ad hoc interest groups, etc. What we need is a tool that will allow groups to form around goals in a manner that is much simpler than current project management tools. Instead of assigning tasks why can't we volunteer for them? I envision a hybrid wiki for developing group purpose and a goal tracker for staying on track. Agile scholarship here we come.
• Link it all up to a service like Twitter or Plazes to up the serendipity quotient.

Then all we need is some angel investor to fund a new institute a la TechStars and Bob's your uncle.

I can feel myself being pulled to the dark side of philosophy through this semester's classes and readings. I'm starting to think about causality. (mock horror)

In my STS class we just finished reading “The Strong Programme in the Sociology of Knowledge” by David Bloor. Bloor lists four conditions for an explanatory theory of science. It must explain the cause of beliefs, be impartial to the truth or falsity of beliefs, be symmetrical and use the same explanatory framework for true and false beliefs, and be reflexive or capable of being applied to sociology as well as science or any other human knowledge seeking/creating endeavor.

From such tiny seeds spring massive controversies.

The fundamental nut of confusion is what does it really mean to say that a belief is “socially constructed.” The scientific realist balks at the suggestion that science is controlled by the dirty biases of our human intellect. He wants to have a pure science that reaches for the truth no matter the circumstances.

The interesting turn that Bloor makes in his essay is to ask why the philosophers of science are willing to accept an explanation that science seeks after true beliefs when science gets something right, but then turn to socio-political explanations when scientists get something wrong. Shouldn't there be some kind of symmetry between the two explanations? Isn't the cause of our success the same as our cause the cause of failure?

Consider evolution. We tell ourselves that Darwin was a brilliant observer of the natural world and thus, the discoverer of evolution. He revealed the truth of the natural world to our eyes. Contrast Lysenko who we say was a puppet of the Soviet political system, a perpetrator of falsehood and politicizer of science.

Both Darwin and Lysenko, and any other scientist before or since, were embedded in a particular society and time. They had access to particular resources, financial, social, and technological. Don't all of these things have something to do with whether they succeed or fail?

There's a fear that if we accept a social explanation for scientific success then we will devalue science. I'm in favor of valuing science much more than we do currently. But still the philosopher in me wants to shake science off of its pedestal. It is as much a human endeavor as any other activity we pursue. There are good reasons to protect this human endeavor from political abuse and good reasons to promote it because it produces new knowledge. Even a socially constructed fact can be believed to be true.

A class assignment in 503 asked for the top favorite books and movies. So it here it is for another audience. I notice that the movie list is a lot more variable than the book list. The book list is more stable than the movie list. I've seen so many movies it's hard to remember the ones that really make an impression. It's also a temperament thing. For books I often remember the place and time I was reading them, the connection is more emotional. For movies it's less so.

Books

1. Collected Wallace Stevens. I first really read Stevens in a poetry class at Yale with Harold Bloom, but I had admired him even in high school.
2. Dispossessed by Ursula LeGuin. The SF novel that opened my eyes to gender issues and political anarchism. It gave me a sense that a social system could really be different than the one we inhabit.
3. Finite and Infinite Games by James Carse. I read this in high school and remember referring to it while writing a summer school essay on George Orwell's “Shooting an Elephant.”
4. Zen and the Art of Motorcycle Maintenance by Robert Pirsig. I think I first plowed through this early in high school or late in junior high. The book that made me want to be a philosopher.
5. Godel, Escher, Bach by Douglas Hofstadter. I remember reading this book in the backseat of my mom's car one summer evening around Lake Calhoun in Minneapolis, probably early high school.

Movies

1. Blade Runner. I think I saw this on videotape during high school. I'm not sure if I was attracted by the movie or the Vangelis score.
2. Brazil. Another biting satire that I remember watching with my high school friends. We all acknowledged that “we're all in it together.”
3. Network. Pure scene chewing anger, I love it.
4. Bridge on the River Kwai. The hook that set me onto David Lean and the movie epic.
5. Casablanca. The usual suspects. I saw this again over the holiday break and it astounds me how perfectly polished it is. A gem from the heart of the studio age.

I could easily add more books (Winter's Tale by Mark Helprin, Cloud Atlas by David Mitchell) or movies (anything by Powell and Pressburger), but here's the list as of today.

Scott Page, an economics, political science, and complex systems professor at Michigan spoke to the ICOS seminar today about his new book The Difference: How the Power of Diversity Creates Better Groups, Firms, Schools, and Societies.

His talk straddled the line between pop social science books, like Blink or The Wisdom of Crowds, and an academic talk. The line tended to the more pop economics end of things than the academic, but I enjoyed it. He began by reviewing a lot of the material I've encountered before in Sami's prediction market course or Surowecki's book.

The interesting part came near the end when he introduced some of the theorems he has created to explain the wisdom of crowds. One of them is the Diversity Prediction theorem: Collective Error = Average Individual Error - Prediction Diversity. Diversity in this case is dependent on negative correlation between the predictors in the group. In the book he describes this as the projection property.

Understanding the Projection Property requires careful thought. It says that if two people look at different attributes of the same perspective (that is, different dimensions), and if the task is to predict success or failure, or any other binary outcome such as good or bad, or yes or no, then when one person is correct, the other person is less likely to be correct. Thus, they're better at collectively predicting than they'd be if they got independent signals...

The projection property implies that crowds containing people who look at diverse attributes will be wise. Unfortunately, this insight cannot be leverage as much as we might hope. The dimensionality of the perspective defines the number of nonoverlapping projection interpretations. A perspective that creates a five-dimensional representation of an event or situation can support at most five nonoverlapping projection interpretations.

The dimensions he's talking about here are akin to the criteria used to judge a situation. Page used the example of hiring someone based on charisma or experience in his presentation today. In that case charisma or experience are the dimensions.

I noticed one thing during the presentation that might be an interesting way to look at the problem of gathering collective knowledge. Page mentioned the game show “Who Wants to be a Millionaire?” in his talk, echoing a portion of Surowecki's book. According to Surowecki, polling the crowd for an answer was correct 95% of the time, and calling a friend (expert in Page's terminology) was only correct 2/3 of the time.

Page and Surowecki assume that the person who was called was an expert. Functionally the 'call a friend' was equivalent to consulting an expert. But when I watched the show the contestant was much more likely to call a friend, someone they knew personally and who they guessed might have some information to help answer the question. I think calling these people experts is a stretch.

The real problem with the 'call a friend' option is that it in only goes one-degree into the network of people known by the contestant. If there were a way for the contestant to know the skills and knowledge of the people who were two or more degrees beyond her friends then the 'ask an expert' possibility would work much more effectively.

And suddenly the problem turns itself into the old knowledge management conundrum of expertise finding. I guess there is some connection in all this information science stuff after all.

Here's a list of interesting items that crossed my radar in the last week. I don't have much to add to what's said below, this is more a manner of keeping track of my interests at this point in time.

The future of humanity

• Bruce Sterling dialoging at the WELL
• Joe Bageant laying into American complacency. More from an anti-consumerist point of view.
• James Howard Kuntsler laying into American energy complacency

Some thoughts on interesting technology

• a review of Macintosh software development
• Robert Cringely predicts 2007
• Mozilla heading toward an information broker instead of a browser
• Setting up OpenID with your blog
• an introduction to microformats

Creating a new educational/learning world.

• Bill Tozier points toward something as yet undefined, but very interesting
• Dave Pollard translates the long tail into the center and the edge
• Jakob Nielsen reminds us how difficult it is to get people to participate online, or elsewhere
• Christopher Sessums on skills for participation

Rhetorics of the internet age

• Clancy Ratliff on John Logie's book about peer-to-peer rhetoric, the actual book
• Mrke Bernstein on hypetext literature, an area that I'd like to investigate in more depth.
• Reports on a survey of teens about online privacy
• The copyright regime takes some hits

Creativity

• A Typology of Cognitive Pleasures via Ming the Mechanic

Philosophy of Information and Technology

• Ted Byfield gives a brief history of information

Personal Productivity and other improvements

• Guy Kawasaki on the art of schmoozing
• Gurteen knowledge log on networking

Science, mostly astronomy

• 3d mapping of dark matter, the bad astronomy blog reports in more detail on the same study
• a short bit on computational complexity and big-O notation

Via Geomblog I found an interesting article by Malcolm Gladwell about intelligence gathering, the Enron scandal, and our human habits of thought. Gladwell frames the problem using the language of puzzles and mysteries. Puzzles depend on missing data. If only there was a bit more information then the puzzle could be solved. Mysteries depend on the interpretation of information. Getting more information is rarely enough to solve a mystery, too many ambiguities remain for tidy conclusions to be reached.

If things go wrong with a puzzle, identifying the culprit is easy: it’s the person who withheld information. Mysteries, though, are a lot murkier: sometimes the information we’ve been given is inadequate, and sometimes we aren’t very smart about making sense of what we’ve been given, and sometimes the question itself cannot be answered. Puzzles come to satisfying conclusions. Mysteries often don’t.

Daniel Kahneman and Jonathan Renshon have another interesting article about the psychological biases that predispose leaders to believe hawks over doves when it comes to deciding on foreign policy.

Social and cognitive psychologists have identified a number of predictable errors (psychologists call them biases) in the ways that humans judge situations and evaluate risks. Biases have been documented both in the laboratory and in the real world, mostly in situations that have no connection to international politics. For example, people are prone to exaggerating their strengths: About 80 percent of us believe that our driving skills are better than average. In situations of potential conflict, the same optimistic bias makes politicians and generals receptive to advisors who offer highly favorable estimates of the outcomes of war. Such a predisposition, often shared by leaders on both sides of a conflict, is likely to produce a disaster. And this is not an isolated example.

In fact, when we constructed a list of the biases uncovered in 40 years of psychological research, we were startled by what we found: All the biases in our list favor hawks. These psychological impulses—only a few of which we discuss here—incline national leaders to exaggerate the evil intentions of adversaries, to misjudge how adversaries perceive them, to be overly sanguine when hostilities start, and overly reluctant to make necessary concessions in negotiations. In short, these biases have the effect of making wars more likely to begin and more difficult to end.

via Matthew Yglesias

The question I'm left with is whether we can build information tools that will help with these problems. Tools are useful but a lot of the things that really need to change are attitudes and realizations, things that go back even further into our earliest education and perhaps into our biology. Changing these things is hard.

Listening to The Whirling Dervish from the album “World Trio” by Dave Holland, Mino Cinelu, Kevin Eubanks