but there's always a computable number with less than epsilon error, so would this ever actually matter?
Yes. There are programs that can neither be proven to halt nor to not halt. The knowledge that these do not halt would need to be specifically programmed in, and the K-complexity would increase for each one.
On the other hand, it's not exactly known how common these are. They might be pretty rare.
In any case, you could change it from shortest computer program to shortest definition, and thus get definable numbers instead of computable ones.
Here's something I've been wondering about, in the context of Solomonoff induction and uncomputable sequences.
I have a device that is either a halting oracle, or an ordinary Turing machine which gives the correct answer to the halting problem for all programs smaller than some finite length N but always outputs "does not halt" when asked to evaluate programs larger than N. If you don't know what N is and you don't have infinite time, is there a way to tell the difference between the actual halting oracle (which gives correct answers for all possible programs) and a "fake" halting oracle which starts giving wrong answers for some N that just happens to be larger than any program that you've tested so far?
The Kolmogorov complexity of an uncomputable sequence is infinite, so Solomonoff induction assigns it a probability of zero, but there's always a computable number with less than epsilon error, so would this ever actually matter?