If it's worth saying, but not worth its own post (even in Discussion), then it goes here.

Open Thread, June 2-15, 2013
New Comment
436 comments, sorted by Click to highlight new comments since:
Some comments are truncated due to high volume. (⌘F to expand all)Change truncation settings
[-]gwern740

Per a discussion on IRC, I am auctioning off my immortal soul to the highest bidder over the next week. (As an atheist I have no use for it, but it has a market value and so holding onto it is a foolish endowment effect.)

The current top bid is 1btc ($120) by John Wittle.

Details:

  1. I will provide a cryptograpically-signed receipt in explicit terms agreeing to transfer my soul to the highest bidder, signed with my standard public key. (Note that, as far as I know, this is superior to signing in blood since DNA degrades quickly at room temperature, and a matching blood type would both be hard to verify without another sample of my blood and also only weak evidence since many people would share my blood type.)
  2. Payment is preferably in bitcoins, but I will accept Paypal if really needed. (Equivalence will be via the daily MtGox average.) Address: 17twxmShN3p6rsAyYC6UsERfhT5XFs9fUG (existing activity)
  3. The auction will close at 4:40 PM EST, 13 June 2013
  4. My soul is here defined as my supernatural non-material essence as specified by Judeo-Christian philosophers, and not my computational pattern (over which I continue to claim copyright); transfer does not cover any souls of gwerns in altern
... (read more)

Just increased my subjective probability that John Wittle is Satan.

I am really disappointed in you, gwern. Why would you use an English auction when you can use an incentive-compatible one (a second price auction, for example)? You're making it needlessly harder for bidders to come up with valuations!

(But I guess maybe if you're just trying to drive up the price, this may be a good choice. Sneaky.)

[-]gwern390

(But I guess maybe if you're just trying to drive up the price, this may be a good choice. Sneaky.)

Having read about auctions before, I am well-aware of the winner's curse and expect coordination to be hard on bidding for this unique item.

Bwa ha ha! Behold - the economics of the damned.

[-]StJohn160

Sorry to ruin the fun but I'm afraid this sale is impossible. Gwern lacks the proprietary rights to his own soul. As the apostle St Paul writes in his letter to the Corinthians (chapter 6), "Or know you not, that your members are the temple of the Holy Ghost, who is in you, whom you have from God; and you are not your own? For you are bought with a great price. Glorify and bear God in your body." It clearly states that "you are not your own" which at least applies to baptized Christians (and as a confirmed Catholic, it may even apply to a higher degree). Unless gwern provides some scriptural basis for this sale, it cannot proceed. Even when Satan tempted Christ, the only proferred exchange was worship in return for temporal power. There are no cases (even hypothetical ones) of a direct sale of one's soul in the Church's Tradition.

In exchange for ruining this sale, I'll pray for your soul for free.

6Plasmon
That's because Satan knows there's no such thing as a soul, and he is disinclined to lie.
2gwern
This seems inapplicable to me; I haven't agreed to sell my soul yet, and so far the bidding hasn't been too active so it will hardly be for 'a great price'.
5[anonymous]
I believe the "great price" is referring to God sacrificing Jesus to redeem the souls of all humanity, including (presumably) you. But I'm hardly a biblical scholar; see below, lol.
0gwern
Sure, but presumably I still have control over the disposition of my soul, otherwise that's basically a Calvinist theology, no?
1[anonymous]
I'd like to ruin gwern's sale too, but my misspent youth as a philosophy major just came back to haunt me. [EDIT: This paragraph is completely wrong; see below. The end of 1 Corin 6:19 does not say "you are not your own"; it literally says "and [it] is not your own" (= καὶ οὐκ ἐστε ἑαυτῶν) with an omitted subject. The only real possibility is the subject of the previous phrase, which you rendered as "your members." (= τὸ σῶμα ὑμῶν) I find this problematic (and not "clearly stated"), because σῶμα means both the Church as a group (usually in the form, "the body of Christ") and the physical body, as it does in e.g. Mat 10:28: "Do not be afraid of those who kill the body but cannot kill the soul. Rather, be afraid of the One who can destroy both soul and body in hell."] Since in context 1 Corin 6:12-20 is about sexual immorality, I find the latter interpretation more compelling. Regarding the Catholic tradition, time was when the Church claimed the authority to discharge sin from the soul in exchange for money.

The end of 1 Corin 6:19 does not say "you are not your own"; it literally says "and [it] is not your own" (= καὶ οὐκ ἐστε ἑαυτῶν)

You are wrong about this - here's the inflection of the word: http://en.wiktionary.org/wiki/%CE%B5%E1%BC%B0%CE%BC%CE%AF#Ancient_Greek

"ἐστε" is second person plural ("you are") NOT third person singular ("it is").

5[anonymous]
Oh, blast. My biblical Greek is obviously too old. Retracting paragraph.

so souls likely constitute a lemon market.

applause

[-][anonymous]130

you still benefit infinitely by bargaining him down to an agreement like torturing you every day via a process that converges on an indefinitely large but finite total sum of torture while still daily torturing you & fulfilling the requirements of being in Hell.

A tactic that almost definitely should be referred to as "Gabriel's Horn."

Note that if you can get a high price from Satan on your own soul (e.g. rulership of a country), this is a no-lose arbitrage deal since souls are fungible goods.

8Decius
Reference?
5Jayson_Virissimo
I tried to find that "all are equal in the eyes of God" verse, but apparently there isn't one. Curious.
-11Eliezer Yudkowsky
1FriendlyButConcerned
0James_Miller
Because they don't exist?
[-][anonymous]100

My soul is here defined as my supernatural non-material essence as specified by Judeo-Christian philosophers, and not my computational pattern (over which I continue to claim copyright); transfer does not cover any souls of gwerns in alternate branches of the multiverses inasmuch as they have not consented.

What? This is lame. The definition of the soul as used by 16th century Catholic theology, which is friendly to information theory, is clearly the common sense interpretation and assumed among reasonable people. Sure some moderns love the definition you use but they are mostly believers of moralistic therapeutic deism, one hardly needs more evidence of their lack of theological expertise.

1gwern
None of that seems true to me, although I'll admit I don't know what revolution happened in the 1500s in Catholic theology re souls.
[-]Shmi100

I certify that my soul is intact and has not been employed in any dark rituals such as manufacturing horcruxes; I am also a member in good standing of the Catholic Church, having received confirmation etc. Note that my soul is almost certainly damned inasmuch as I am an apostate and/or an atheist, which I understand to be mortal sins.

Not sure how much I can trust the word of a damned. After all, lying is no more of a mortal sin than apostasy. And for an atheist there is no extra divine punishment for lying.

4gwern
Ah, but can we take your word for it? IIRC, you are one of my fellow damned...
2Shmi
I am not sure. I have never been baptized, so where my soul ends up depends on whether exclusivism, inclusivism, conditionalism or universalism is true.
0gwern
I'm pretty sure that by Catholic dogma, you would count as definitely damned due to lack of baptism and knowing of the Church but refusing to convert to it.

One person who did this years ago spun the event into a book, a popular blog, and endless speaking gigs.

[-]gwern120

That's an interesting comparison, but I'm selling my soul, and it looks like he was just selling his time:

Mehta, an atheist, once held an unusual auction on eBay: the highest bidder could send Mehta to a church of his or her choice. The winner, who paid $504, asked Mehta to attend numerous churches, and this book comprises Mehta's responses to 15 worshipping communities, including such prominent megachurches as Houston's Second Baptist, Ted Haggard's New Life Church in Colorado Springs, Colo., and Willow Creek in suburban Chicago.

3lukeprog
Oh right, I was misremembering what he did.
9Kaj_Sotala
Huh, reading this made me realize that there's apparently still a small bit of my brain that doesn't alieve in atheism. For a moment I considered whether I should try to get some profit out of selling my soul as well, and then felt uncomfortable over the idea, thinking "I should hold onto it, just in case..."
0gwern
I actually really decided to do the auction when I thought about the topic and realized that it didn't bother me at all. Might as well profit from my lack of belief/alief.
8Douglas_Knight
The quitclaim doesn't help here. It merely quits your claim, which is relevant if ownership is disputed, but it doesn't give any more rights to the buyer than to anyone else (just more documentation of the quit). You should have been suspicious when taterbizkit mentioned that you can sell quitclaim deeds for a single item to multiple buyers.
6gwern
After some unfortunate imperial entanglements, the sale has been completed: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 I, the undersigned, do irreversibly sign over to the purchaser possession of my soul, karmic balance, or whatsoever ontologically basic mental system might underly my own consciousness, in the event that we live in a functionally dualistic or monistically spiritual universe, for the sum of $121 USD on this June the Eleventh 2013 (with delayed payment calculated at 9% interest compounded annually), to be paid to the PayPal account identifiable as gwern0@gmail.com If our universe is reductionistic, and yet, some alien agency continues to compute the mental processes of our minds after death, giving us pleasurable or painful experiences based on how optimized our earthly behavior was as measured against some criteria, then I explicitly sign over possession of the weight of all the actions I took in life over to the purchaser, with the resolution of any problems to be determined by the aforementioned alien agent. This day being the Seventh of June 2017, I now accept payment of $144 from the purchaser such that the above terms are binding. - -- Gwern Branwen gwern0@gmail.com Bitcoin hash: 00000000000000000055b0bee08bfeb235bc60bc22a27951501d78b10883484a -----BEGIN PGP SIGNATURE----- iQIcBAEBCgAGBQJZOEisAAoJEH3Oo4eJxYjMVIQP/1q4oFISyLS+4YAPpe5vmZu+ 8TWNxX8G00Ix1qduEo/Okg6YTE8jdJk8wIwuuMGp2EeVIdUD8ffPz75TUiCzBpGq NpOPh/UW8Z6QPcx0FkkBBwG9UhHqlyWWhGmZtHMt3LYDbsLlD8AsXfDF2/leUxqQ dxeNi7NPUImI6vPsCMRV+vRoKzAvd1odzeO0vK0KmWgjQUw5jeEHf7liCr383DpY R7rsx8GgCV/2PesGzCJSmPjtJmEKTeZspQTXW9D2bHRpkozhYgxZ7PKhVFv5rLUd twaJhM83LmiA+y7lZ7kA2DFADtgwhLGXhSsdW1SWxrOq6tg4VxYN8HvfoL+MTdlK ZvwwuWmk7fzKXGdOrz2PKrirGv1WqUeAQTDsRUeh0dPI2BxDafLSQymIa2M9jVro LusLVs6caDGLMBPTitB/gbijtCaKqEApM1nqlAehJ3gqIzOGu98o1CW1+HpQ+qI6 e6uq3/6C34eb5E9R69NHl/jgiXHlppvCYKwA8f82k44aPiTHjgPU5hNExwKZWx8i 2VjQCnYc8hO614VmYKMnLKXbPT0jBmsA6hYR6sV8hKhoJ7Bj299DFxyw471l7HUW 7GoO4UkN9r1F8JTn/0H+xSbOA
6TheOtherDave
conversely, if Satan insists on my soul, I can let Satan have my soul and use yours instead.
5Gnnthkcclqnrx
FYI, according to galactic law, transactions like this are valid only to the extent that the implicit metaphysics of the contract is correct. If you wish to guarantee the property rights of your soul's new owner, you should add a meta clause indicating valid interpretive generalizations of content and intent.
6gwern
I'm afraid I can't afford a barrister admitted to the Trantor bar to look over the contractual details, but thanks for the advice.
4JohnWittle
Heh, I would have bid 0.5btc if I had known I would be the only bidder...
7Vaniver
This makes this exchange all the more amusing.
2FourFire
I'm obviously missing something, but tally ho, I'll find out eventually!
8Vaniver
A second-bid auction is one where all bidders submit their maximum willingness to pay, and then the bidder willing to pay the most pays what the second-highest bidder was willing to pay. An English auction is where bidders submit bids which they will have to pay, with the idea that once the second-highest bidder will stop raising the bid once they pass their threshold. There's a lot of theoretical work showing that second-bid auctions are all-around more efficient. English auctions can encourage the highest bidder to overbid, and the winner's curse refers to the phenomenon that the winner of an auction is generally the person who overestimated its value by the most. Second bid auctions mitigate that by making them pay only the second highest estimate. If JohnWittle is the only bidder in the auction, then in a second-bid auction he would receive gwern's soul for free, but because this is an English auction, he has to pay his full bid, and so loses out for dramatically overestimating its market value- like gwern planned all along!
2badger
I'm don't specialize in auctions, but this sounds wrong. A second-price auction and an English auction are strategically equivalent in most formal models. Nearly all auctions yield identical revenue and allocations when bidders are risk-neutral expected utility maximizers with independent values. Experimentally, the second-price auction tends to generate more revenue than an English auction, at least in the case of private values. With common or correlated values (where the winner's curse shows up), I'd think sealed bid auctions would lead to more winner overbidding than English or Dutch auctions. In these cases though, you really don't have to worry about efficiency since everyone values the item equally.
0Vaniver
I should have been clearer by 'all-around'; I meant that the incentives are lined up correctly, the costs are lower (every person only needs to submit one bid, and does not need to expend any effort monitoring the auction), gets exact results without requiring massive numbers of bids, and more information is conveyed by the end of the auction.
0TheOtherDave
Well, yes, technically that's true... but what prevents/discourages gwern (or his accomplice) from submitting an $N-1 bid (where N is the current sole bid amount)?
5Vaniver
Typically, second-bid auctions are sealed, and all opened at once at the end of the auction, so it won't be known that JohnWittle has bid, or how much he has bid, until the auction is over.
0TheOtherDave
Ah. (nods) That makes sense.
0gwern
If I were going to do that, I would simply have set a reserve price.
0TheOtherDave
Not the same thing, surely? Submitting an N-1 bid causes the top bidder to pay effectively their bid... in effect turning a second-bid auction into an English auction as defined above. Setting a reserve price sets a floor that has no relationship to the top bidder's bid. But sure, the fact that you didn't set a reserve price also suggests that you wouldn't take advantage of this loophole in your counterfactual second-bid auction.
4Larks
What if these are in fact the same thing, in extension if not intention? Then you would be selling your computational pattern, in contradiction with
2gwern
I think that's unlikely enough that I'm willing to risk a tort of fraud if that turns out to be the case and I cannot convey my soul without also selling my personal copyright.
3listic
You definitely should auction it off in other places, where prospective buyers value such things much higher.
6gwern
What other forums might value my soul? As a purchase, it's really most useful for atheists willing to do a simple expected-value calculation and hedge against a tail risk (theism); but for most people, buying a soul is largely otiose.
7TheOtherDave
Wait.... it seems you're suggesting that the expected value of a soul to an atheist exceeds the otiosity threshold. Did I read that right? I'm interested in your reasoning, if so. Either way: the expected entertainment value to me of purchasing your soul far exceeds the expected value of the soul itself, and I suspect that's not uncommon, so I doubt the theological implications are a primary factor.
1gwern
It depends on one's subjective uncertainty. I know there are atheists who have been persuaded by visions or Pascal's wager that they were wrong, so the risk would seem to be real, and given the stakes, $120 seems like chump change for insurance - even if you try to defeat a Pascal's wager by bounded utility, the bound would have to be extremely large to be plausible...
9Douglas_Knight
If atheists thinks that there's a small chance that they will turn into theists and be glad to be in possession of a spare soul, then they must think that theists value spare souls. So it would seem more valuable to theists, who don't have to multiply the value of the transaction by the small chance. There are some differences between typical theists and the hypothetical atheist-turned-theist. In particular, the theist has had a lifetime to keep a clean soul. But many theists think they do a bad job. If the spare soul has tail risk value to an atheist, it should have more value to the bad theists. The other difference is that the atheist is not a believer at the time of the transaction. Perhaps the belief of the theist makes it a greater sin to trade in souls. But it seems like a lot of details have to go right for it to be a better deal for the atheist than the theist.
0TheOtherDave
Mm. Are you suggesting that the subjective uncertainty of a typical atheist on this question causes expected value to exceed the otiosity threshold? Or merely that there are some atheists for whom this is true? I'll agree with the latter. Though, thinking about this, surely this would be much more likely for theists, no? So wouldn't the maximum expected value of your soul likely be higher, thereby securing you a higher sale price, in a theist community? (Preferably one with a sense of humor about theology.)
3CronoDAS
Hindus and some other groups may disagree with that. ;)
[-]Decius110

If you can find evidence that they are correct, you could have a fraud claim. However, the contract defines the soul being sold as that described by the Judeo-Christian philosophers.

2Zaine
What do you intend to do with your soul(s) as defined by other schools of philosophy? By Plato's theory of Ideal Forms, selling your soul would be tantamount to selling bits of the gods - and man has no claim to the gods. I'd advise against this lest you wish to become fate-brothers with Prometheus.
0[anonymous]
What? Citation needed.
0Zaine
Phaedo 80b.
2[anonymous]
Ah, Socrates supposes there that the soul is "like the divine" as opposed to the body which is like mortal things. He means that the soul is in the class of things that are unchanging, immutable, invisible, and grasped by the intellect rather than the senses, He doesn't say anything about the soul being a 'part of the gods'. And it doesn't sound like he's thinking of anything like the Prometheus myth, given the things he associates with the soul (ideal, invisible, immutable, etc.). If you asked Plato about selling your soul, I think he would think you were just being silly.
0Zaine
If something was divine, then it was under the domain of the gods; I was making a simple extrapolation.
0[anonymous]
Yeah, but that's not a sound inference, given the context. No mention is made there of the gods, and the context pulls wide away from reading 'divine' in terms of traditional Greek mythology. I see no reason to think Socrates (or Plato) thinks any of that stuff was real.
1Decius
Are you accepting bids in things other than currencies commonly used for exchange? I would like to offer a finely crafted narrative instead of bitcoins.
2gwern
Hm, is your narrative so compelling that I would accept jam tomorrow instead of bitcoin today?
0thomblake
Upvoted for the multilayered pun
0Decius
I offer no guarantees regarding the quality, completeness, or any other details of said narrative (save that it will be a narrative, delivered within 90 days of acceptance of terms, with payment in full due immediately on receipt), although I will accept your input, if you want me to, on length, theme, setting, genre and/or other details. As for the relative value of narratives and btc, I can say only that I have not written for any commonly recognized currency. Accepting this offer would subject you to a considerable amount of downside risk, as well as a considerable amount of upside risk. However, people who auction their soul are not typically averse to these types of risk.
4gwern
Mm, I'm afraid that due to the hyperinflation over the past few decades of narrative and subsequent debasement (>3.2m on FanFiction.net alone), I can't accept any amount of it without guarantees of its quality. Nothing personal - it's the law.
0Decius
What would you accept as sufficient evidence of quality?
6[anonymous]
A Hugo Award, I presume.
6gwern
Or a Nebula, Locus, or World Fantasy Award. I'd also accept a Nobel or Man Booker (for magical realism).
0Decius
Which one do you want? I can have a crack team of ninja liberate it from the current owner and deliver it to you, but that will cost significantly more than your soul.
2gwern
Well then, I'm afraid we would be unable to reach a mutually beneficial agreement - I would be better off retaining my soul under such a sale.
1Decius
You could also earn or steal your own frikkin' literature award. The typical narrative written by a winner of a high-prestige award is worth significantly more than a few btc in straight commercial value. I acknowledge that my narrative will very likely have negative commercial value (it would take more work to sell it than it would be purchased for), or I would be selling narratives and they would be too valuable to me to offer to you. The thing is, I wouldn't offer anything based on its cash value, because I value your soul only slightly more than you do. My hope was to find something that you would prefer to btc as the price of your soul. A narrative set to music might be particularly appropriate, since it would allow you to say that you sold you soul for a song.
0gwern
While that is tempting, I am sufficiently amused that I will be able to say I sold my soul for bitcoins and - diminishing returns - selling it for a song isn't amusing enough to sell it on the cheap to you. Anyway, it would violate the terms I've already set.
0Decius
Fair enough. Enjoy your bitcoin.
0elharo
I flashed back to Bill Wilingham's Proposition Player. Highly recommended for an amusing fantasy take on this particular deal.
0Michelle_Z
I laughed out loud when I read this. I'm not incredibly surprised someone would bid, but at the same time, disappointed.
[-][anonymous]200

I am confused. This Washington Post article appears to describe a preliminary study which suggests that politics is less of a mindkiller if you ask people to bet money on their beliefs.

http://www.washingtonpost.com/blogs/wonkblog/wp/2013/06/03/if-you-pay-them-money-partisans-will-tell-you-the-truth/

And I am confused because what appear to be my attempts to find the paper resulted in two papers with entirely different abstracts. And papers. Example:

Abstract 1:

"Our conclusion is that the apparent gulf in factual beliefs between members of different parties may be more illusory than real."

Abstract 2:

"Partisan gaps in correct responding are reduced only moderately when incentives are offered, which constitutes some of the strongest evidence to date that such patterns reflect sincere differences in factual beliefs."

http://huber.research.yale.edu/materials/39_paper.pdf

http://themonkeycage.org/wp-content/uploads/2011/04/bullockgerberhuber.pdf?343c0a

I realize the dates on the papers are different, but the shifts seem very dramatic. Thoughts?

0Lumifer
Maybe this will help? http://marginalrevolution.com/marginalrevolution/2013/06/55494.html

I scraped the last few hundred pages of comments on Main and Discussion, and made a simple application for pulling the highest TF-IDF-scoring words for any given user.

I'll provide these values for the first ten respondents who want them. [Edit: that's ten]

EDIT: some meta-information - the corpus comprises 23.8 MB, and spans the past 400 comment pages on Main and Discussion (around six months and two and a half months respectively). The most prolific contributor is gwern with ~780kB. Eliezer clocks in at ~280kB.

6jefftk
What about for the site overall?
6sixes_and_sevens
This was my eventual plan, but I haven't settled on a general corpus to compare it to yet.
4Kawoomba
Can you comment on your methodology - tools, wget scripts or what?
2sixes_and_sevens
Scraping is done with python and lxml, and the scoring is done in Java. It came about as I needed to brush up on my Java for work, and was looking for an extensible project. I also didn't push it to my personal repo, so all requests will have to wait until I'm back at work.
2Richard_Kennaway
Yes please. I have no idea what they will look like.
2sixes_and_sevens
suffering -> 25.000 god -> 24.508 does -> 24.383 causal -> 21.584 np -> 21.259 utility -> 20.470 agi -> 20.470 who -> 20.169 pill -> 19.353 bayesian -> 18.965 u1 -> 17.567 The word 'who' seems to come up a lot for the contributors at the more prolific end of the scale. I don't have a satisfactory answer why this should be the case. Your contribution comprises ~170kB of plain text.
0[anonymous]
If I'm counting the replies correctly, nine respondents requested them so far. I'd like my word values. Thank you!
2sixes_and_sevens
political -> 28.733 power -> 27.093 moldbug -> 26.135 structural -> 24.192 he -> 24.082 reactionary -> 23.480 blog -> 21.973 good -> 21.373 social -> 20.470 his -> 20.470 very -> 20.169 Your contribution is ~167kB.
0ArisKatsaris
May I have mine? Thanks.
0sixes_and_sevens
moral -> 35.017 thread -> 34.250 bob -> 25.163 preferences -> 24.383 eu -> 23.739 column -> 23.537 matrix -> 23.419 mugging -> 22.367 pascals -> 21.479 lord -> 19.515 eg -> 19.266 Your contribution to the corpus is ~100kB.
0FiftyTwo
An alternative would be to ask people for donations to Against Malaria Foundation or your preferred charity.
0Dorikka
I'd like mine, please.
4sixes_and_sevens
gvrq -> 9.457 puppies -> 8.784 cute -> 7.141 creprag -> 7.119 gb -> 6.901 rewind -> 6.305 fvatyr -> 5.100 deck -> 4.838 stuff -> 4.816 vf -> 4.739 boom -> 4.221 As mentioned to other respondents, rot13 really messes with TF-IDF. I'm still not sure of the best way to deal with this.
0Douglas_Knight
If someone uses rot13, that is a highly informative. Is there any principled reason to like quoted words showing up, but not liking rot13? Anyhow, I think the disappeal of rot13 for TF-IDF is that it seems like a lower level feature than words. In particular, it is wasteful for it to show up more than once, if you're only doing top 11. In some sense, I think the reason that the low level feature of rot13 is mixing with the high level feature of words is that you've jumped to the high level by fiat. Before looking a word frequency, you should look at letter frequency. With a sufficiently large corpus, rot13 should show up already there. I doubt that the corpus is big enough to detect the small usage by people here, but I think it might show up in bigrams or trigrams. I don't have a concrete suggestion, but when you look at bigrams, you should use both corpus bigrams and document letter frequencies to decide which document bigrams are surprising.
0sixes_and_sevens
You've already surmised why rot13 words are undesirable. Just to check, are you suggesting I use n-gram frequency to identify rot13 words, or replace TF-IDF with some sort of n-gram frequency metric instead?
0Douglas_Knight
You could use TF-IDF on n-grams. That's what I was thinking. But when I said to combine combine the local n-gram frequencies and the global n+1-gram frequencies to get a prediction of local n+1-gram frequencies to compare against, you might say it's too complicated to continue calling it TF-IDF. If all you want to do is recognize rot13 words, then a dictionary and/or bigram frequencies sound pretty reasonable. But don't just eliminate rot13 words from the top 11 list; also include some kind of score of how much people use rot13. For example, you could use turn every word to 0 or 1, depending on rot13, and use TF-IDF. But it would be better to score each word and aggregate the scores, rather than thresholding. What I was suggesting was a complicated (and unspecified) approach that does not assume knowledge of rot13 ahead of time. The point is to identify strange letter frequencies and bigrams as signs of a different language and then not take as significant words that are rare just because they are part of the other language. I think this would work if someone wrote 50/50 rot13, but if the individual used just a little rot13 that happened to repeat the same word a lot, it probably wouldn't work. (cf. "phyg") There are two problems here, to distinguish individuals and to communicate to a human how the computer distinguishes. Even if you accept that my suggestion would be a good thing for the computer to do, there's the second step of describing the human the claim that it has identified another language that the individual is using. The computer could report unusual letter frequencies or bigrams, but that wouldn't mean much to the human. It could use the unusual frequencies to generate text, but that would be gibberish. It could find words in the corpus that score highly by the individual's bigrams and low by the corpus bigrams.
0Douglas_Knight
mine, please.
0sixes_and_sevens
sats -> 22.952 htt -> 22.810 sat -> 22.157 princeton -> 21.356 mathematicians -> 17.903 crack -> 16.812 harvard -> 16.661 delete -> 16.563 proofs -> 15.745 graph -> 15.565 regressions -> 15.301 Your corpus comprises ~77kB of plain text.
0Vaniver
I'd like mine, please!
4sixes_and_sevens
because -> 41.241 p -> 38.129 should -> 34.016 sat -> 33.974 much -> 33.113 cholesterol -> 33.056 evidence -> 32.444 iq -> 32.092 comments -> 31.454 scores -> 30.690 clear -> 28.899 Your contribution comprises ~284kB of plain text, and is the thirteenth-largest in the corpus.
2Vaniver
Thanks! Interestingly, the only one of those that I recognize as clearly one of my verbal quirks is "clear," which I use a lot in "it's not clear to me that ...", but it barely made it onto the list. I participate in most of the discussions on intelligence testing, so it's no surprise that "sat," "iq," and "scores" are high. "Cholesterol" seems likely to be an artifact from a single detailed conversation about it, and then apparently I like words like "because," "should," and "much" more than normal, which is not that surprising given my general verbosity. I know I use the word "evidence" more than the general population, but am surprised I use it that much more than LW, and "comments" is unclear. Probably meta-discussion?
4sixes_and_sevens
Most incidence of "comments" seems to be in the context of moderator actions. There are 44 occurrences in your contribution to the corpus, which is around 50,000 words. As for "evidence", there are 70 occurrences in 50,000 words. So on average, every 715th word you say in comments is "evidence".
0satt
Ooh, go on then.
2sixes_and_sevens
phd -> 34.505 teleology -> 25.661 maitzens -> 20.402 neutron -> 19.191 fusion -> 17.502 causal -> 17.267 argument -> 16.222 turtle -> 16.137 greenhouse -> 15.736 p1 -> 15.353 might -> 15.353 Your contribution comprises ~116kB.
0satt
Haha, I should've foreseen "maitzens", "causal", "argument" & "turtle" showing up there. (I'm lucky your corpus didn't go back far enough to capture this never-ending back-and-forth, otherwise my top 10 would probably be nothing but "HIV", "AIDS", "cases", "CDC", "Duesberg", "CD4", and such.) Thanks for running the numbers.
0TheOtherDave
Sure, why not? Thanks!
0sixes_and_sevens
x -> 98.136 confidence -> 87.600 value -> 66.797 agree -> 65.843 endorse -> 63.750 ok -> 60.507 said -> 59.640 evidence -> 54.869 say -> 54.185 bamboozled -> 53.497 values -> 53.122 Your contribution comprises ~420kB of plain text, and is the fifth largest in the corpus.
0arundelo
Cool! This (judging the relevance of words in documents in a corpus and analogous problems) is a subject I muse about sometimes. Thanks for introducing me to TF-IDF. I'd like my top scoring words please.
2sixes_and_sevens
comte -> 17.852 m1 -> 12.664 grumble -> 9.813 altruism -> 8.787 rotating -> 8.442 olive -> 8.150 comtes -> 8.025 m -> 7.383 workshop -> 7.157 egoistic -> 6.916 happiness -> 6.475 Your contribution comprises ~21kB of plain text.
0Kaj_Sotala
Curious to hear mine.
2sixes_and_sevens
intelligence -> 17.119 machine -> 15.353 environments -> 15.052 reference -> 13.546 machines -> 12.304 views -> 12.253 legg -> 12.252 friedman -> 11.417 papers -> 10.792 we -> 10.536 exercises -> 9.532 Your contribution to the corpus amount to ~47kB of plain text. For reference, Eliezer is ~190kB and gwern is ~515kB. The scores are unadjusted for document size and not amazingly meaningful outside of this specific context.
0Kaj_Sotala
Huh, that seems different from what I'd have expected - but then again, I'm not sure of what I would have expected. Thanks.
4sixes_and_sevens
I've just fixed a bug in my scraper that was causing it to abandon 25% of the corpus. This has ended up tripling your contribution. Some new values for you: agi -> 37.328 intelligence -> 22.367 moral -> 21.010 agis -> 20.087 eea -> 18.647 takeoff -> 17.500 credences -> 17.108 machine -> 16.902 our -> 16.222 environments -> 15.919 deer -> 15.761 This retains a similar "flavour" to the previous set, (AGI and ev-psych). The best way I've found to interpret it is "what sort of words describe what I use Less Wrong to talk about?" As an interesting side-note, rot13 really messes with TF-IDF.
4Kaj_Sotala
Okay, that feels like it makes more sense. I'm a little confused about the "deer", though.
6sixes_and_sevens
Blame this comment.
2Kaj_Sotala
Hah, okay.
0Richard_Kennaway
You're not distinguishing original from quoted text, then?
0sixes_and_sevens
It's not obvious to me that I should. TF-IDF is about identifying key terms in a document. Quoted text counts towards that.
0Richard_Kennaway
That depends on what "the document" is. Everything appearing in a posting by a given author, or all of the text written by a given author?
0sixes_and_sevens
"The document" is my wild sample that I've gone out and caught. TF-IDF tells me what it's broadly about. For this purpose, quoted text provides useful information. If I want to infer personal facts about the author (beyond "what are the key terms in the posts they write"), it would make sense to weight original text higher than quoted text, but it would also make sense to use something other than TF-IDF for that purpose.

On BBC Radio 4 this morning I heard of a government initiative, "Books on Prescription". It's a list of self-help books drawn up by some committee as actually having evidence of usefulness, and which are to be made available in all public libraries. They give a list of evidence-based references.

General page for Books on Prescription.

The reading list.

The evidence, a list of scientific studies in the literature.

I have not read any of the books (which is why I'm not posting this in the Media Thread), but I notice from the titles that a lot of them are based on Cognitive Behavioural Techniques, which are generally well thought of on LessWrong.

The site also mentions a set of Mood-boosting Books, "uplifting novels, non-fiction and poetry". These are selected from recommendations made by the general public, so I would say, without having read any of them, of lesser expected value. FWIW, here's the list for 2012 (of which, again, I have read none).

1[anonymous]
I notice that almost all of those books are about things that are considered "mental problems" (the exceptions being chronic fatigue, chronic pain, and relationship problems, which are nevertheless specific problems). So if a self-help book isn't about a particular problem (like How to Win Friends and Influence People and The Seven Habits), or the problem it talks about isn't primarily psychological (like Getting Things Done), then it won't appear on that list regardless of how good it is. (Stating my opinions here so that you won't have to guess: My brother, who seems quite sensible and whom I admire very much, states that all three of the books mentioned here are very good. Getting Things Done taught me one extremely useful lesson, probably among the top five most useful things I have ever learned. I have little evidence, apart from this stuff, that any of these books are useful.)

Iain Banks is dead.

"They speak very well of you".

-"They speak very well of everybody."

"That so bad?"

-"Yes. It means you can´t trust them."

8FiftyTwo
Fuck cancer. Fuck mortality. We must work faster.

Improving my social skills is going to be my number one priority for a while. I don't see this subject discussed too much on LW, which is strange because it's one of the biggest correlates with happiness and I think we could benefit a lot from a rational discussion in this area. So I was wondering if anyone has any ideas, musings, relevant links, recommendations, etc. that could be useful for this. Stuff that breaks from the traditional narrative of "just be nicer and more confident" is particularly appreciated. (Unless maybe that is all it takes.)

Optional background regarding my personal situation: I am a 19 yo male (as of tomorrow) who is going to enter college in the fall. I'm not atrociously socially inadept, e.g. I can carry on conversations, can be very bold and confident in short bursts sometimes, I have some friends, I've had girlfriends in the past. However, I also find it very hard to make close friends that I can hang out with one on one, I sometimes find myself feeling like I'm taking a very submissive role socially, and I feel nervous or "in my head" a lot in social interactions, among other things. Not to be melodramatic, but I find myself wishing a decent amount that I had more friends and was more popular.

Improving my social skills is going to be my number one priority for a while. I don't see this subject discussed too much on LW, which is strange because it's one of the biggest correlates with happiness and I think we could benefit a lot from a rational discussion in this area.

Discussion on lesswrong on that subject would most likely not be rational. Various forms of idealism result in mind killed advice giving which most decidedly is not optimized for the benefit of the recipient.

Stuff that breaks from the traditional narrative of "just be nicer and more confident" is particularly appreciated. (Unless maybe that is all it takes.)

Get out of your house, go where the people are and interact with them. Do this for 4 hours per day for a year (on top of whatever other incidental interactions your other activities entail). If "number one priority" was not hyperbole that level of exertion is easily justifiable and nearly certain to produce dramatic results. (Obviously supplementing this with a little theory and tweaking the environment chosen and tactics used are potential optimisations. But the active practice part is the key.)

4gothgirl420666
I agree that when social skills are usually discussed, various forms of idealism tend to result in mind killed advice. The standard set of advice in particular seems to mostly ignore the fact that a) status exists, i.e. it is very possible to be liked and not respected, and sometimes the latter overpowers the former and b) some people genuinely have large personality flaws that make them unpleasant to be around. I was hoping LessWrong would be able to avoid this idealism, as they do in most other areas, which is why I posted here. Do you think that LessWrong would be worse than average in this regard? Why? And do you think there is anywhere I could have a rational discussion about this stuff? Like I said in another post in this thread, I don't think it's at all a given that if you socialize enough, you will eventually develop good social skills, and I think that reading a bit of stuff on the subject in the last month helped me about as much as all the social experiences I've had in the last year. But something about the idea of making it a priority to spend x amount of time a day specifically seeking out social interactions makes sense and is appealing to me. I don't know if four hours a day is the right amount - I will have to experiment, but I can very much see myself implementing something like this. One problem with widely recommending this is that it seems to me like many, if not most people are not at all in a position to reliably be able to follow this advice. But I imagine someone with low to moderate social skills on a college campus probably can.
0Vaniver
Sure, but it should be ~30 minutes of reading a day and ~4 hours of interaction a day. Practice is what leads to skill development, and unpleasantly enough, only hard practice (i.e. focusing on the parts you're bad at, not the parts you're good at) really counts.
8drethelin
Practice Practice Practice Practice Practice. You have to go out of your way to hang out with people to get any good at being fun to hang out with. WARNING: This does not mean you have to spend time at loud parties or bars or clubs. While they pretend to be areas for socializing, they're not really. It's one thing if you enjoy dancing or drinking, but places that are less loud and crowded are a lot better for conversation.
4gothgirl420666
I've done this and it didn't really work. Maybe it worked a little, but not at a very fast rate. To be honest, I think reading a small amount of social skills stuff and thinking about how to solve the problem a little helped much more than all the "practice" I've done in the last year or so. Obviously you can't take this to the extreme and expect that you can instantly go from Michael Cera to Casanova just by sitting alone reading stuff and watching videos in your room, but I don't think the statement "If you spend enough time in social interactions, you will inevitably develop good social skills" is at all true either. I kind of despise the former and love the latter. :\
8Viliam_Bur
Did you try dancing lessons? I hated dancing before I learned it, but I love it now. I am very bad at "learning by copying others", but with good explicit education I became a decent dancer. (Note: Almost everyone adviced me against explicit learning, because they said it wouldn't be "natural" or "romantic". I ignored all this advice, and now no one complains about the result. Contrary to predictions, learning the steps explicitly helped me to improvise later. Seems like people just have a strong taboo about applying reductionism to romantic activities like dancing.)
2Nisan
Interesting; no one has ever told me that dancing lessons are a bad idea. I think we live in very different cultures. (Other things you have said in the past have also given me his impression.)

No, no, no, this was a bad explanation on my part. No one told me that dancing lessons are bad idea per se... only that my specific learning style is.

This is what works best for me: Show me the moves. Now show me those moves again very slowly, beat by beat. Show me separately what feet do; then what hands and head do. Tell me at which moment which leg supports the weight (I don't see it, and it is important). When and how exactly do I signal to my girl what is expected from her. (In some rare situations, to get it, I need to try her movements, too.) I still don't get it, but be patient with me. Let me repeat the first beat, and tell me what was wrong. Again, until it is right. Then the second beat. Etc. Then the whole thing together. Now let's do the same thing again, and again, and again, exactly the same way. Then something "clicks" in my head, and I get the move... and since that moment I can lead, improvise, talk during dance, whatever. -- As a beginner I was blessed with a partner who didn't run away screaming somewhere in the middle of this. Later my learning became faster, partially because I learned to ask the proper questions. And I had a good luck to dancing tea... (read more)

0A1987dM
Now, that's about the only possible way to learn to play anything non-trivial on instruments such as the guitar; therefore, these people 1. believe that all guitar music is ugly and robotic, or 2. have no idea of how people learn to play, or 3. are confused and/or talking through their asses (e.g. some part of them deep down is saying ‘people who cannot learn to dance the way I did don't deserve to get the social status I got from it’) (not necessarily with probabilities within an order of magnitude of each other).
4Viliam_Bur
I completely agree with you (which is why I persisted in my learning style). From my experience it seems to me many people are confused like this. Possible explanation: We learn some things by copying or early in childhood, and we learn some other things explicitly. I guess this makes many people think that skills are divided to "explicitly teachable" and "explicitly unteachable", using some heuristics, such as: "if it is usually learned at school, it is teachable", "if I tried to learn it and failed, it is unteachable", "it is teachable only if I perfectly understand how it works", etc. It probably adds to confusion that we don't see how other people learned their skills. Similarly to attribution fallacy, if we see someone good at doing X, it is easier to assume that it is a part of their nature, not a learned skill. (And those people may support us in this opinion, for example because it discourages the competition.) Seems to me this is pretty frequent in art. Also, sometimes the idea of "unteachable skill" is a good excuse for not learning and doing something. Even those people who learned e.g. playing guitar may not propagate the idea automatically to other aspects of their lives.
0Richard_Kennaway
Sometimes people don't see how they themselves learned something. When you ask them, they confabulate empty phrases like "it's a knack", or "eventually you just get it", or the like. They generally suck at explaining. So, ignore them and move on.
0A1987dM
I was assuming that those people had themselves learned to dance at some point, so unless it was a very long time ago and/or they suck at introspection they knew how they did it. If you were talking about people who didn't themselves know how to dance, then replace ‘people who cannot learn to dance the way I did don't deserve to get the social status I got from it’ with ‘I'm jealous those people can dance and I can't, but I can't be bothered to learn it myself, so in order to put them down I'll tell them that their grapes are sour’.
4Viliam_Bur
Maybe there are two learning styles -- copying and explicit -- each of them having their set of advantages and disadvantages. (Perhaps an analogy to System 1 and System 2.) Learning by copying is faster and it does not require cooperation from the person you copy. On the other hand, copying is imperfect, and you cannot copy what you don't see. Learning explicitly is slower and requires a good explanation; which requires a good introspection from the person who explains. So maybe this is an instance of "the last will be first". -- People who are good at learning by copying, use learning by copying as their favorite learning style. People who are bad at learning by copying can compensate by focusing on explicit learning. Under these assumptions, the "copying" people have a fast start, because many activities are simple and can be learned by copying. Then when it comes to more complex activities, they usually continue copying, get some mediocre results, and stop there. And even there, they probably get those mediocre results faster than an "explicit" person. -- They really believe that learning by copying is superior, because this is what worked for them. Learning explicitly is just a strange ritual done at school; and I suspect that even there they try to copy the teachers. On the other hand, "explicit" people learn slowly and are completely dependent on good learning materials. Sometimes the good materials are available, and allow them to reach mastery in complex things. The whole school system is designed for this. Sometimes the materials are unavailable or misleading (e.g. because the topic is mindkilling), and they are lost. These are the "book smart" people. -- They believe in explicit learning, because this is what worked for them. These are just extreme descriptions, I guess most people use learning by copying in some areas and explicit learning in other areas. They may have an explanation about which style is better in which situation. There are things th
0gothgirl420666
Sorry for only commenting on the irrelevant taboo topic you touched on, but this is interesting to me. I have been reading some PUA stuff lately and it seems to me that the whole point is that it is not describing something that ordinary humans learn naturally, but instead prescribing something extraordinary that you can do to set yourself apart from the crowd in order to attract the hottest girl in the club that every other guy in there is hitting on. And even then it only works via the law of averages, and requires one to override one's natural intense aversion to rejection in order to pursue a more rational strategy adapted for a modern world in which you can talk to someone once and never see them again. Am I wrong about this?

These days PUA refers to so many things that I need to be more specific. The sources that helped me were "The Mystery Method" by Mystery, "How To Become An Alpha Male" by Carlos Xuma, "Married Man Sex Life" by Athol Kay. I would also recommend "The Blueprint Decoded" by RSD.

Yes, there are many sources that only tell you "do this, do that, and if it does not work, just do it again". I guess this is what most customers want: "Don't bother me with explanations, just give me a quick fix!" This is how most people approach everything. Well, if there is a demand for something, the market will provide a product. And these days it is a huge business. Ten years ago, it was more like geeks experimenting and sharing their results and opinions... a bit similar to Quantified Self today, just less scientific, and sometimes more narrowly focused.

Overcoming aversion to rejection, doing many approaches to convert given rates of success into greater absolute numbers, doing something extraordinary to stand out of the crowd... those are the fixes. Applied incorrectly they could be even harmful. (Receiving a lot of rejection can make you more r... (read more)

7gothgirl420666
[deleted]
8A1987dM
A more accurate way of putting that is that the man is the first to break plausible deniability. If you also take into account non-verbal, indirect signals (where if the recipient isn't interested they can just pretend to not notice and nothing bad happens), most of the times the very first move is the woman's, both according to this report about Britain and in my experience in both Italy and Ireland: I can't say I can recall ever getting a positive reaction from approaching a woman who wasn't already smiling at me. Now, a guy who has good social skills but poor introspection may only approach women who are smiling at them but not be consciously aware that he's preselecting women that way; likewise, a socially savvy but not introspectively savvy woman may not be consciously aware that she's smiling at the guy she likes; as a result, it feels to them like it's the man who's initiating the interaction, which I guess is the main cause of that confusion.
0gothgirl420666
Interesting. Although "if she's smiling at you, she likes you" seems like it wouldn't hold true when you're trying to flirt with acquaintances.
0A1987dM
With people you already know, the kinds of indirect signals (where if the recipient isn't interested they can just pretend to not notice and nothing bad happens) are different (and not all of them are entirely non-verbal), but otherwise the same kind-of applies.
2gothgirl420666
Do you know what these indirect signals are? This seems like useful information.
0