I think this is a really important question, and something I think a lot about. This is a pretty consistent theme of my blog. Some posts I've written that might help (If you'll forgive the blatant self-promotion):

My favourite: Become a person who actually does things - make 'being an agent' part of your identity, and something to aim for for its own sake. Do something because it feels agenty, not necessarily because it's obviously the best thing to do. It's much easier to get good at being agenty about the right things once you've built the habit in the first place

Notice when you're procrastinating on something, and develop a toolkit for things to do in the moment to ensure you take action

A lot of my failure to be agenty comes from being risk-averse and fearing downsides. Learn to look past the safety of the "default" action of doing nothing, and to get excited about small chances of massive upsides

Have a regular time in your life to reflect on things, and give yourself prompts to notice what you could be being more agenty about. Make sure that noticing and correcting this kind of thing is a default part of your life, rather than something that needs effort and inspiration. Make agency the path of least resistance

If you feel bottlenecked by creativity or ideas, set a 5 minute timer and use the time pressure and urgency to spur yourself into doing things

I sometimes roleplay as someone role playing as myself, then take the action that I would obviously want to take, e.g. "wow sleeping regularly gives my character +1 INT!" and "using anki every day makes me level up 1% faster!"

How to be an agent: Abstain from all of passive activities including watching YouTube, playing videogames, reading blogs, following the news, abnegation drugs, etc. Music, podcasts and audiobooks are okay because they don't displace you from doing things. Put all your effort into not doing things. Do not compel yourself to do anything. Put zero effort into doing things unless you just feel like it. If you're not doing passive activities then either you're doing active activities (which makes you an agent) or you're doing nothing (which makes you a yogi). It's really hard to be a yogi so most people turn into agents within one week of abstaining from passive activities.

• Make sure you don't have undiagnosed ADD and spend ten years of your life going from one self help technique to the next and failing to get your executive brain to use any of them.

• Also, virtuous cycles are incredibly powerful for me. It's much easier to start a big project when I have a few victories fresh in my mind than when I don't. So I try to go for quick wins whenever I have free time. Small home improvement projects work well for me and also make my surroundings nicer.

• Avoid clutter. This can double as a way to procrastinate, but I've found that I can focus much less in a messy place.

• Have agenty friends. If not able to: Watch interviews with agenty people or read fiction about them.

• For longer projects, make meditating on them a habit. I usually contrast where I am with where I want to be and why. This helps avoid shiny new things getting in the way.

I think it helps to create specific, interlaced frameworks, algorithms, schedules, and plans for more forms of behavior.

Although this is boring and complicated to read about, here is an example:

• I have a spreadsheet, which I use as a schedule. I'm a full-time student, and on that spreadsheet, I list every assignment for the entire quarter. I color-code it as I complete things.
• But in addition, I have a schedule of work periods and break periods. In the morning I work 100 minutes on, 20 minutes off; in the afternoon, 25 minutes on, 5 minutes off. I also take a 20-second look away from the screen break every 20 minutes.
• I have a list of permitted and banned activities during breaks, and types of chores that are only permitted during work time or only permitted during breaks.
• I've also divided forms of work into categories: school, paid jobs, side projects.

I allow myself to modify this framework freely, and I don't follow it on all days. But when I do, it's a very helpful guide that improves my productivity as well as my sense of wellbeing.

Yet it is very far from a natural/intuitive way of operating. There are lots of timers. I take breaks when I don't think I need them, and choose not to do what I normally would.

As another example, I've also crafted an explicit set of commenting and karma awarding standards for participating in LW conversation.

When I'm not working, I don't use these frameworks. They're a tool I pick up when I want to be more agenty, and put down when I don't.

To me, what it means to be an agent is that you have an explicit framework that's unambiguous in your mind, which dictates your behaviors and the way you allocate time. Just creating a framework, adding to it, and playing with it until you've shaped it to be a net improvement is a good thing to try.

Being agenty should be easier and feel more natural than not being agenty if you're doing it right. We're in an unnatural system. If you bring your evolved patterns to an unnatural system, they'll grate against each other. If instead you can create a framework that helps guide your behaviors to fit more smoothly with the unnatural system, everything can work much better.

Don't overgeneralize.  Be specific in the things you address, and identify distinct blockers to your intended activities.  

"Akrasia" is a syndrome loosely identified by the results, not a single addressable dysfunction.

On the extreme end basically guaranteed to work: use arduino to build a system that detects when you are doing the tasks you wish to do, then use that system to condition access to food and sex on you actually doing those things. This will require fixing a whole bunch of loopholes in your system, if hacking your system is easier than just doing your tasks, you will hack it. 

In the same vein but less extreme: place a scale underneath your treadmill to detect when you are using it and have an alarm sound daily at the time when you should be running, the alarm doesn't stop until you go on the treadmill. If you want to study, isolate one corner of your room with a room-divider that you bolt to your walls with a timed lock, the idea is that you can't quit your desk for a pre-determined amount of time.

What's worked for me isn't focusing on agency per se. I've had more success from focusing on my deeper desires (for which "agency" is often instrumental) and figuring out how to get them. Sometimes those plans run into psychological barriers. When that happens, I'll do whatever it takes to overcome or dissolve those barriers—rationality techniques, therapy techniques, pure willpower, esoteric philosophy, etc etc. After repeating this a bunch I ended up more proactive than before because there were fewer mental barriers between me and taking the "agenty" action when it happened to be a good idea.

Like, I wasn't thinking "I should be more agenty, I'll go [organize a speaker series | raise tens of thousands of dollars for weirdo projects | change my interpersonal demeanor | solve an intellectual problem that no one I know can answer] to practice agency." Rather, I found myself in situations where things like that were good ways to get what I wanted but I was too averse to actually do it, then wrestled with my soul until I could do it anyway. (Sometimes this step takes two hours, sometimes it takes six months.) Each step unlocked more of a general willingness to do similar things, not just the narrow ability to do that one thing.

Of the people I know who seriously follow an approach like this for at least a couple years, about 50% wind up notably more effective than their peers and about 10% wind up insane.

One take-away I really liked about The Replacing Guilt Sequence was the idea that---instead of choosing between actions, you should think about the world-state that each action results in, then choose which of those world-states that you like more. But that's just me, and your mind probably reacts a bit differently.

For the situation in which you find yourself---in which you don't really know what you want to do. You just want to do something, I'd recommend the lessons from this post from the sequence.

It seems to me that the listless guilt usually stems from not doing anything in particular. I'm not sure how to remove that feeling of guilt in people who aren't doing anything in particular. But if they shift the guilt to being guilty about not doing one thing in particular, then I have some tools that might help.

Might be helpful:

• Notes on Ambition
• Notes on Industriousness

If there is only one thing you would reliably do, it probably makes sense to establish a regular time to reflect on things. Not sure what is the optimal amount. One hour a week? Maybe make it a walk in nature/park, with pen and paper, so that you are not tempted to cut it short or start doing something else. Don't use this time to force yourself to do anything. (Don't do anything that would make you hate this ritual.) Just think, observe, and dream.

Establish good habits. (This sounds like a contradiction: "habits" vs "agenty". The idea is that you are agenty about choosing and updating your habits, but you use the habits as the raw power to achieve the goals. For example, if the goal is writing a novel, the agenty thing is to establish the habits of regularly reading and writing, and reflecting on progress.) Regularity beats volume. If you can make yourself do one push-up consistently every morning, it is relatively simple to increase the amount later.

For inspiration, talk to other smart and agenty people. (May be difficult to find them. Perhaps I am prejudiced but I believe that "read/watch inspiring blogs/videos" is a wrong answer: famous bloggers will optimize for making you an addicted reader and potential customer, not for making you more agenty. Look for a relationship where you talk with someone as a friend, not where someone is trying to sell you things.) This gives you inspiration and peer pressure. Overcome your ego: people who are ahead of you are the best to learn from, but it may make you feel like a loser. (Maybe too much ahead is not good, because they would be too difficult to copy.)

Perhaps try meditation; one of the first achievements is the ability to turn off unwanted distractive thoughts when they come. Then, whenever you face an unpleasant task, precommit to work on it seriously for five minutes, and use the skill to turn off distracting thoughts during that time period. (You may be surprised to find out that it is actually quite easy to continue doing the task when the five minutes are over. Yes! But don't push yourself. Precommit to the five minutes only; everything afterwards is optional.)

Read Don’t Shoot The Dog by Karen Pryor to get a "gears model" how conditioning works. (Sometimes things that work for some people don't work for others, but for me this was the best book on the topic, and it explained so many important things, including some common mistakes people make. If there is one book people would ever choose to read based on my recommendation, it is this one.)

Mental model that works for me is that of Self-Organization (from cybernetics and complexity theory). Use interactions to enforce redundancy of potential command(See McCulloch). This way you still retain the agency as the command is initiated by you, but the emergence of order(i.e., the execution of that task in our case) can be orchestrated by changing the nature of interaction. One example would be to decentralise the decision making process when you sense the beginning of dissolution of willpower: commitment contracts(eg. beeminder), substitute activity(eg. using stairs to compensate for lack of exercise), automating tasks that can be automated, fractal attention(eg. picking up a different book when you are bored with the book, instead of stopping reading altogether) etc.