I'd like to condition the responses by elaborating on what I currently do and why I think having a body of work to reference would be beneficial.

This all started a couple months ago when I realized that most of internet usage revolved around Reddit/Quora + Wikipedia + arXiv. I use the internet to have questions answered, to reference authoritative information, and to explore developments in various research interests.

This equilibrium was disturbed by the invention of GPT-2. I started a project of writing a journal in conjunction with this "super"-autocomplete algorithm and realized that part of the reason I thought it sounded so great was because I was prompting it with things you might find on Reddit/Wikipedia/arXiv. This is kind of hard to explain, basically I awoke to this gestalt that the way I was thinking was being affected by the content I was consuming. Obvious in hindsight, but at the time quite a shocker.

I researched some ways to start curating my own content and found Pocket. After that, things started taking off. I currently use a combination of Pocket/GPT-2/Google to manage my curation of internet content. To constrain information overload, I generally use a question -> hypothesis -> research/experiment workflow. Sometimes getting the right question is hard so I'll use GPT-2 to try and "super"-autocomplete my way into a phrase that has potential. After that I try to google the question/phrase that popped in the first step. However, GPT-2 really has been sending me all over the internet so it's quite difficult to relate things together or to evaluate the quality of the information I'm receiving.

I think having some sort of book organizing the different "dimensions" of internet usage would be useful. Being able to have a tangible organization layout of the available tools would help me select ones that useful for whatever I'm interested in. At the moment I have only a few; search, save, "super"-autocomplete. Any and all references are useful, but the more in depth the better. Thanks!


New Answer
Ask Related Question
New Comment

3 Answers sorted by

It's a great reference! I see this as an edge case where you effectively have an answer, but you need the correct pointer to get there. I think a compliment to this is the case where you need good leads for finding an answer. For example, a journalist might need good leads to write an expose.

One part I'd highlight is the case-studies section: https://www.gwern.net/Search#case-studies [https://www.gwern.net/Search#case-studies]

Sound a bit like Howard Rheingold’s Net Smart.

Is Net Smart very practical? The introduction sounds more theoretical and generic, and it's a good 7 years old now. (I noticed when I saw references in that link to long-defunct websites like CureTogether.)

I think the more practical ideas in it (custom RSS readers?) are outdated.
2 comments, sorted by Click to highlight new comments since: Today at 8:09 PM

How exactly do you implement using GPT-2 for autocompletion? This usage is new to me.

In theory? You just generate a few random samples with the current text as the prefix and display them. In practice, there's already tools to do this: Talk to Transformer does autocomplete. Even better, IMO, is Deep TabNine for programming languages, trained off Github.

New to LessWrong?