**Next message:**hal@finney.org: "Re: How You Do Not Tell the Truth"**Previous message:**hal@finney.org: "Re: How You Do Not Tell the Truth"**In reply to:**hal@finney.org: "Re: How You Do Not Tell the Truth"**Next in thread:**Samantha Atkins: "Re: How You Do Not Tell the Truth"**Reply:**Samantha Atkins: "Re: How You Do Not Tell the Truth"**Messages sorted by:**[ date ] [ thread ] [ subject ] [ author ]

hal@finney.org wrote:

*>
*

*> The part about Robin's paper (http://hanson.gmu.edu/deceive.pdf) that
*

*> I have a hardest time understanding is the discussion of common priors.
*

Hm. Well, I understood it perfectly. Therefore, you should take this

into account in deciding whether or not the paper makes sense, unless you

don't think we have common priors.

Essentially, Robin's paper gives a rigorous mathematical proof that for

two people to (a) disagree and (b) maintain their disagreement after

interaction, one or both of the parties must believe that they are more

likely to be rational than the other person. This does not necessarily

prove irrationality in *all* cases but it proves irrationality for *most*

cases. If we take into account the evolutionary-psychology arguments,

Robin's paper makes a strong case for a built-in irrationality factor

common to all humans.

If a 100,000:1 genius is interacting with a 10,000,000:1 genius, but

neither of them knows the other's percentile, both will rationally assume

that they are more likely to be rational than the other person. However,

Robin's paper does prove that in *most* cases, rather than in the rarer

instances where two geniuses unknowingly interact, people must be

overestimating their own rationality relative to others, or else must not

be using rigorous Bayesian reasoning with respect to what they are

licensed to conclude from their own thoughts.

-- -- -- -- --

Eliezer S. Yudkowsky http://singinst.org/

Research Fellow, Singularity Institute for Artificial Intelligence

**Next message:**hal@finney.org: "Re: How You Do Not Tell the Truth"**Previous message:**hal@finney.org: "Re: How You Do Not Tell the Truth"**In reply to:**hal@finney.org: "Re: How You Do Not Tell the Truth"**Next in thread:**Samantha Atkins: "Re: How You Do Not Tell the Truth"**Reply:**Samantha Atkins: "Re: How You Do Not Tell the Truth"**Messages sorted by:**[ date ] [ thread ] [ subject ] [ author ]

*
This archive was generated by hypermail 2b30
: Mon May 28 2001 - 10:00:02 MDT
*