For context, I've been teaching myself programming for about 10 years, and I'm currently a professional programmer.
The most important thing in programming is that you find something that you enjoy. There's enough variety in terms of languages and problems out there that you're bound to find something that works for you. Once you find something that's fun (and therefore easier) for you, it won't be too hard to bootstrap into more complicated languages and problems.
As far as problems go, start with something simple, and work your way up. An excellent way to do this is Project Euler, which many people here have recommended as well. I think it would fit particularly well in your case, since many of the problems on that site are mathematical. Another great thing about Euler is that it is language-independent. Make sure that after you solve a problem, you read the solutions other people have posted. You can find examples of a lot of different languages and styles.
As to language choice, I'd go through some tutorials online, and see what makes sense to you. One you might try is Why's Poignant Guide to Ruby. Maybe some other people have other good recommendations for other languages. Try the first few problems in several languages, and see what makes the most sense to you. I like Java and Python. Personally, I'd avoid LISP, Scheme, Haskell, and other more academic languages unless you're particularly ambitious, and/or you know people who use them.
An important skill in programming I think most people forget is this: Research skills. I've got a lot of experience programming, but I come across an enormous amount of things that I simply don't know how to do. Get used to using google to find answers. It's much faster than asking people or looking it up in a book.
I'd recommend against installing Unix/Linux unless you're already familiar with it. Learning programming and learning Unix are both non-trivial tasks. While I agree that Unix is generally a better environment for programming, I think it's better to focus on one thing at a time. Optimizing your programming environment is secondary to using an environment you're comfortable with.