Joseph Van Name

Joseph Van Name2d*

Yes. When we take convex combinations of finitely many point mass measures, the integral is just a sum. I use the sum of finitely many elements for ease of calculations, but to prove theorems, I should use measures for full generality.

The idea of finding an object $A$ along with distinct local optima $G_{A} (x_{1}), \dots, G_{A} (x_{n})$ with $n$ maximized looks like an interesting problem to work on. I have not worked on this kind of objective before, but I can certainly try this, as I have a few ideas of how to do this. This might work better for discrete optimization problems though since I cannot think of a good way to use gradient updates to produce new local optima. In... (read 563 more words →)

Joseph Van Name5d

For experiments, I just used a convex combination of point mass measures for $μ$ where the point masses are generated uniformly at random (though I might get something more complicated if I tried evaluating the integrals). I then attempted to find multiple local maxima by the usual gradient ascent. If I always end up with the same local maximum, I presume that there is only one local maximum even though I have no mathematical proof that this is the case.

I am redoing the experiments and the only way I can get pseudodeterminism to fail is by using real inner product spaces instead of complex inner product spaces and by setting n=1 (and in this... (read more)

Joseph Van Name5d*

Yes. It seems like to get pseudodeterministic AI, we will need to rebuild AI from the very beginning, and I am not sure that it will all work. For example, pseudodeterminism is harder to attain with stochastic or mini-batch gradient descent, so one might need to use all the training data whenever one updates the weights. I have so far been able to get pseudodeterministic multi-layered models for solving classification problems, word embeddings for NLP, models that are measurements of security of block ciphers such as the advanced encryption standard (the models evaluating the AES are very easy to train), and other things. I have not been able to make pseudodeterministic version... (read more)

Joseph Van Name6dQuick Take

For machine learning, it is desirable for the trained model to have absolutely no random information left over from the initialization; in this short post, I will mathematically prove an interesting (to me) but simple consequence of this desirable behavior.

This post is a result of some research that I am doing for machine learning algorithms related to my investigation of cryptographic functions for the cryptocurrency that I launched (to discuss crypto, leave me a personal message so we can discuss this off this site).

This post shall be about linear machine learning models. Actually, we are using quantum operators, so they are more sophisticated than your logistic regression models, but they are still... (read 811 more words →)

Replying toLiteracy is Decreasing Among the Intellectual Class

Joseph Van Name3mo*

Literacy is Decreasing Among the Intellectual Class

You are siding with evil because you yourself are evil. My anger against people like you is righteous. If you are not convinced, it is simply because you have been completely consumed by your own evil. Universities promote violence. I know. I was a professor. But you just want me to be violently attacked and injured or killed.

~~UNIVERSITIES MUST BE REJECTED FOR PROMOTING VIOLENCE!~~

P.S. Only commenting and responding to my non-technical posts where I call out universities for their problem just proves my point. Face it. You are stupid, and universities only pretended to educate you.

I am striking this out because it is better if I instead made only mathematics posts on this site relevant to AI safety.

-7

Replying toLiteracy is Decreasing Among the Intellectual Class

Joseph Van Name3mo*

Literacy is Decreasing Among the Intellectual Class

I was a professor, so I know that universities promote violence, so don't even try that bullshit on me. People hate universities because universities are absolutely horrendous and extremely unprofessional. Until universities apologize for their extremely low standards and horrible behavior, we should MOCK all people with degrees from universities and regard them as evil worthless people. People hate me for bringing this up because most people with college degrees are scumbags who are afraid to admit that they are far stupider (and more evil too) than people who did not waste their money and time in college.

~~I DO NOT NEED TO VISIT A UNIVERSITY TO KNOW HOW IT IS LIKE SINCE~~... (read more)

-40

-2

Replying toLiteracy is Decreasing Among the Intellectual Class

Joseph Van Name3mo*

Literacy is Decreasing Among the Intellectual Class

We should no longer consider people with degrees or affiliated with universities as being 'intellectual' in any way whatsoever because universities promote violence and refuse to improve their horrendous behavior.

I am striking this out because it is better if I instead made only mathematics posts on this site relevant to AI safety.

-19

Replying toWhy Not Just Train For Interpretability?

Joseph Van Name3mo

Why Not Just Train For Interpretability?

The use of something like L1 regularization to achieve sparsity for inherent interpretability may just make things worse; a fixation on L1 regularization may lead people in the wrong direction. To avoid fixation, we should take a step back and look at the big picture. Occam's razor suggests that we should look for simple (and creative) solutions instead of over-engineering solutions when the entire foundation is inadequate.

In order to obtain inherent interpretability, the machine learning model needs to behave in a way that is interesting to mathematicians. By piling on tweaks such as a lot of L1 or L0 regularization for sparsity, one is making the machine learning model more complicated. That... (read more)

Approximating arbitrary complex-valued continuous functions

Joseph Van Name

3mo

In machine learning, one often wants to uniformly approximate an arbitrary continuous function arbitrarily well using polynomials, neural networks, or something else. But in the case of complex-valued functions, this is more difficult. For example, the limit of holomorphic functions in the topology of uniform convergence on compact sets is always holomorphic, so holomorphic functions cannot directly approximate non-holomorphic functions. But in this post, I will show you how one can approximate arbitrary continuous functions indirectly from something like a holomorphic function.

I will state and prove the result in the full generality which means I will need to use uniform algebras over compact sets, but I do not need to use the... (read 1214 more words →)

Bitcoin mining is a real-world example of a goal that people spend an enormous amount of resources to attain, but this goal is useless or at least horribly inefficient.

Recall that the orthogonality thesis states that it is possible for an intelligent entity to have bad or dumb goals and that it is also possible for a not-so-intelligent entity to have good goals. I would therefore consider Bitcoin mining to be a real-world prominent example of the orthogonality thesis as it in a sense a dumb goal attained intelligently (though, this example is imperfect).

Bitcoin's mining algorithm consists of computing many SHA-256 hashes relentlessly. The Bitcoin miners are rewarded whenever they compute a suitable... (read 777 more words →)

Using complex polynomials to approximate arbitrary continuous functions

Joseph Van Name

4mo

Machine learning algorithms such as neural networks are supposed to have some sort of universal uniform approximation theorem that shows that they can (at least in principle) learn any possible data set without simply overfitting to the training data.

The standard universal approximation theorem applies to shallow neural networks with arbitrary continuous non-polynomial activation functions. There are also plenty of polynomial approximation results in mathematics, so one should at least in principle be able to train a real multivariate polynomial to model an arbitrary continuous function arbitrarily well using a multi-layered machine learning model. On the other hand, anyone modestly familiar with complex analysis knows that not every continuous function from a compact... (read 1401 more words →)

In this post, we shall go over a way to produce mostly linear machine learning classification models that output probabilities for each possible label. These mostly linear models are pseudodeterministically trained (or pseudodeterministic for short) in the sense that if we train them multiple times with different initializations, we will typically get the same trained model (up-to-symmetry and miniscule floating point differences).

The algorithms that I am mentioning in this post generalize to more complicated multi-layered algorithms in the sense that the multi-layered algorithms remain pseudodeterministic, but for simplicity, we shall stick to just linear operators here.

Let $K$ denote either the field of real numbers, the field of complex numbers, or the division ring of... (read 415 more words →)

I am going to share an algorithm that I came up with that tends to produce the same result when we run it multiple times with a different initialization. The iteration is not even guaranteed convergence since we are not using gradient ascent, but it typically converges as long as the algorithm is given a reasonable input. This suggests that the algorithm behaves mathematically and may be useful for things such as quantum error correction. After analyzing the algorithm, I shall use the algorithm to solve a computational problem.

We say that an algorithm is pseudodeterministic if it tends to return the same output even if the computation leading to that output is... (read 559 more words →)

Spectral radii dimensionality reduction computed without gradient calculations

Joseph Van Name

9mo

In this post, I shall describe a fitness function that can be locally maximized without gradient computations. This fitness function is my own. I initially developed this fitness function in order to evaluate block ciphers for cryptocurrency technologies, but I later found that this fitness function may be used to solve other problems such as the clique problem (which is NP-complete) in the average case and some natural language processing tasks as well. After describing algorithms for locally maximizing this fitness function, I conclude that such a fitness function is inherently interpretable and mathematical which is what we need for AI safety.

Let $K$ denote either the field of real numbers, complex numbers, or the... (read 1590 more words →)

In this post, we shall describe 3 related fitness functions with discrete domains where the process of maximizing these functions is pseudodeterministic in the sense that if we locally maximize the fitness function multiple times, then we typically attain the same local maximum; this appears to be an important aspect of AI safety. These fitness functions are my own. While these functions are far from deep neural networks, I think they are still related to AI safety since they are closely related to other fitness functions that are locally maximized pseudodeterministically that more closely resemble deep neural networks.

Let $K$ denote a finite dimensional algebra over the field of real numbers together with an adjoint... (read 385 more words →)

This is a post about some of the machine learning algorithms that I have been doing experiments with. These machine learning models behave quite mathematically which seems to be very helpful for AI interpretability and AI safety.

Sequences of matrices generally cannot be approximated by sequences of Hermitian matrices.

Suppose that $A_{1}, \dots, A_{r}$ are $n \times n$ -complex matrices and $X_{1}, \dots, X_{r}$ are $d \times d$ -complex matrices. Then define a mapping $Γ (A_{1}, \dots, A_{r}; X_{1}, \dots, X_{r}) : M_{n, d} (C) \to M_{n, d} (C)$ by $Γ (A_{1}, \dots, A_{r}; X_{1}, \dots, X_{r}) (X) = A_{1} X X_{1}^{*} + \dots + A_{r} X X_{r}^{*}$ for all $X$ . Define

$Φ (A_{1}, \dots, A_{r}) = Γ (A_{1}, \dots, A_{r}; A_{1}, \dots, A_{r})$ . Define the $L_{2}$

-spectral radius by setting $ρ_{2} (A_{1}, \dots, A_{r}) = ρ (Φ (A_{1}, \dots, A_{r}))^{1 / 2}$ . Define the $L_{2}$ -spectral radius similarity between $(A_{1}, \dots, A_{r})$ and $(X_{1}, \dots, X_{r})$ by

$∥ (A_{1}, \dots, A_{r}) ≃ (X_{1}, \dots, X_{r}) ∥_{2}$

$= \frac{ρ (Γ (A_{1}, \dots, A_{r}; X_{1}, \dots, X_{r}))}{ρ_{2} (A_{1}, \dots, A_{r}) ρ_{2} (X_{1}, \dots, X_{r})}$ .

The $L_{2}$ -spectral radius similarity is always in the interval $[0, 1]$ . if $n = d$ , $A_{1}, \dots, A_{r}$ generates the algebra of $n \times n$ -complex matrices, and $X_{1}, \dots, X_{r}$ also generates the algebra of $n \times n$ -complex matrices, then $∥ (A_{1}, \dots, A_{r}) ≃ (X_{1}, \dots, X_{r}) ∥_{2} = 1$ if and only if there are $C, λ$ with $A_{j} = λ C X_{j} C^{- 1}$ for all $j$ .

Define $ρ_{2, d}^{H} (A_{1}, \dots, A_{r})$ to be the supremum of

$ρ_{2} (A_{1}, \dots, A_{r}) \cdot ∥ (A_{1}, \dots, A_{r}) ≃ (X_{1}, \dots, X_{r}) ∥_{2}$

where $X_{1}, \dots, X_{r}$ are $d \times d$ -Hermitian matrices.

One can get lower bounds for $ρ_{2, d}^{H} (A_{1}, \dots, A_{r})$ simply by locally maximizing $∥ (A_{1}, \dots, A_{r}) ≃ (X_{1}, \dots, X_{r}) ∥_{2}$ using gradient ascent, but if one... (read more)

In this post, I will post some observations that I have made about the octonions that demonstrate that the machine learning algorithms that I have been looking at recently behave mathematically and such machine learning algorithms seem to be highly interpretable. The good behavior of these machine learning algorithms is in part due to the mathematical nature of the octonions and also the compatibility with the octonions and the machine learning algorithm. To be specific, one should think of the octonions as encoding a mixed unitary quantum channel that looks very close to the completely depolarizing channel, but my machine learning algorithms work well with those sorts of quantum channels and similar... (read more)

Here are some observations about the kind of fitness functions that I have been running experiments on for AI interpretability. The phenomena that I state in this post are determined experimentally without a rigorous mathematical proof and they only occur some of the time.

Suppose that $F : X \to [- \infty, \infty)$ is a continuous fitness function. In an ideal universe, we would like for the function $F$ to have just one local maximum. If $F$ has just one local maximum, we say that $F$ is maximized pseudodeterministically (or simply pseudodeterministic). At the very least, we would like for there to be just one real number of the form $F (x)$ for local maximum $(x, F (x))$ . In this case, all local maxima will typically be related by some sort of symmetry.... (read 715 more words →)

This post gives an example of some calculations that I did using my own machine learning algorithm. These calculations work out nicely which indicates that the machine learning algorithm I am using is interpretable (and it is much more interpretable than any neural network would be). These calculations show that one can begin with old mathematical structures and produce new mathematical structures, and it seems feasible to completely automate this process to continue to produce more mathematical structures. The machine learning models that I use are linear, but it seems like we can get highly non-trivial results simply by iterating the procedure of obtaining new structures from old using machine learning.

I made... (read 540 more words →)

It is time for us to interpret some linear machine learning models that I have been working on. These models are linear, but I can generalize these algorithms to produce multilinear models which have stronger capabilities while still behaving mathematically. Since one can stack the layers to make non-linear models, these types of machine learning algorithms seem to have enough performance to be more relevant for AI safety.

Our goal is to transform a list of $n \times n$ -matrices $(A_{1}, . . ., A_{r})$ into a new and simplified list of $d \times d$ -matrices $(X_{1}, \dots, X_{r})$ . There are several ways in which we would like to simplify the matrices. For example, we would sometimes simply like for $d < n$ , but in other cases, we would like the matrices $X_{j}$ to all... (read 388 more words →)

Replying toCollege Advice For People Like Me

Joseph Van Name10mo

College Advice For People Like Me

I was a professor, so my only advice is to not go to college at all. Colleges are extremely unprofessional and refuse to apologize for promoting violence against me. Since colleges are so busy promoting violence and gaslighting me for standing up for my personal safety, they don't give a shit about your education. Before you all aggressively downvote me for standing up for my own safety, you should learn the FACTS about the situation. Attempts to gaslight me won't work at all because I have trained my mind to resist those who want to harm me.

-22

-7

Joseph Van Name's Shortform

Joseph Van Name

This is a special post for quick takes (aka "shortform"). Only the owner can create top-level comments.

Interpreting a matrix-valued word embedding with a mathematically proven characterization of all optima

Joseph Van Name

In this post, I shall first describe a new word embedding algorithm that I came up with called a matrix product optimized (MPO) word embedding, and I will prove a theorem that completely interprets this word embedding in the simplest case. While it is probably infeasible to mathematically characterize a word embedding with a mathematical proof when the corpus is something that one encounters in practice, this theorem should be a signal that such a word embedding (or a similar word embedding) should be interpretable and mathematical in other ways as well. This theorem also illustrates the way that MPO word embeddings should behave.

Unlike most word embedding algorithms, MPO word embeddings are... (read 3412 more words →)

Interpreting a dimensionality reduction of a collection of matrices as two positive semidefinite block diagonal matrices

Joseph Van Name

Here are some empirical observations that I have made on August 14, 2023 to August 19, 2023 that are characteristics of the interpretability of my own matrix dimensionality reduction algorithm. These phenomena that we observe do not occur on all inputs (they sometimes occur partially); and it would be nice if there were a more complete mathematical theory with proofs that explains these empirical phenomena.

Given a (possibly directed and possibly weighed) graph with $n$ nodes represented as a collection of $n \times n$ -matrices $A_{1}, \dots, A_{r}$ , we will observe that a dimensionality reduction $(X_{1}, \dots, X_{r})$ of $(A_{1}, \dots, A_{r})$ where each $X_{i}$ is a $d \times d$ -matrix (I call this dimensionality reduction an LSRDR) is in many cases the optimal solution to a combinatorial problem for the graph. In this case, we... (read 1426 more words →)

When performing a dimensionality reduction on tensors, the trace is often zero.

Joseph Van Name

In this post, we shall define my new dimensionality reduction for tensors in $V^{\otimes n}$ where $n \geq 3$ , and we shall make an empirical observation about the structure of the dimensionality reduction. There are various simple ways of adapting this dimensionality reduction algorithm to tensors in $V_{1} \otimes \dots \otimes V_{n}$ and even mixed quantum states (mixed states are just positive semidefinite matrices in $V_{1} \otimes \dots \otimes V_{n}$ which trace 1), but that will be a topic for another post.

This dimensionality reduction shall represent tensors in $V^{\otimes n}$ as tuples of matrices $A_{1}, \dots, A_{r}$ . Computer experiments indicate that, in many cases, we have $Tr (A_{i_{1}} \dots A_{i_{m}}) = 0$ whenever $m \neq 0 mod n$ .

If $X$ is a matrix, then the spectral radius of $X$ is the value

$ρ (X) = max {| λ | : λ is an eigenvalue of X} = {lim}_{n \to \infty} ∥ A^{n} ∥^{1 / n}$ .

If $X$ is a matrix, then define the conjugate matrix $¯ ¯¯¯ ¯ X = (X^{*})^{T} = (X^{T})^{*}$ ; this is the matrix obtained from $X$ by replacing each entry with its... (read 773 more words →)

Replying toWork dumber not smarter

Joseph Van Name3y*

Work dumber not smarter

The intersection of machine learning and generalizations of Laver tables seems to have a lot of potential, so your question about this is extremely interesting to me.

Machine learning models from generalized Laver tables?

I have not been able to find any machine learning models that are based on generalized Laver tables that are used for solving problems unrelated to Laver tables. I currently do not directly see any new kinds of deep learning models that one can produce from generalizations of Laver tables, but I would not be surprised if there were a non-standard machine learning model from Laver like algebras that we could use for something like NLP.

But I have to be... (read 2346 more words →)

LESSWRONG
LW

LESSWRONG
LW

Interpreting a dimensionality reduction of a collection of matrices as two positive semidefinite block diagonal matrices

When performing a dimensionality reduction on tensors, the trace is often zero.

Spectral radii dimensionality reduction computed without gradient calculations

Approximating arbitrary complex-valued continuous functions

Joseph Van Name

Joseph Van Name

Approximating arbitrary complex-valued continuous functions

Using complex polynomials to approximate arbitrary continuous functions

Spectral radii dimensionality reduction computed without gradient calculations

Joseph Van Name's Shortform

Interpreting a matrix-valued word embedding with a mathematically proven characterization of all optima

Interpreting a dimensionality reduction of a collection of matrices as two positive semidefinite block diagonal matrices

When performing a dimensionality reduction on tensors, the trace is often zero.

Joseph Van Name

Interpreting a dimensionality reduction of a collection of matrices as two positive semidefinite block diagonal matrices

When performing a dimensionality reduction on tensors, the trace is often zero.

Spectral radii dimensionality reduction computed without gradient calculations

Approximating arbitrary complex-valued continuous functions

Joseph Van Name

Joseph Van Name

Approximating arbitrary complex-valued continuous functions

Using complex polynomials to approximate arbitrary continuous functions

Spectral radii dimensionality reduction computed without gradient calculations

Joseph Van Name's Shortform

Interpreting a matrix-valued word embedding with a mathematically proven characterization of all optima

Interpreting a dimensionality reduction of a collection of matrices as two positive semidefinite block diagonal matrices

When performing a dimensionality reduction on tensors, the trace is often zero.