Math

Update first! This is not matmul on any integer, but rather single digit matmul. I also have to mention that the code and below text was written by Grok 4.5 (Expert).

In the original MFFT post the polynomials were written down to single binary digits. The point of that choice was not aesthetics: once every coefficient is \(0\) or \(1\), a pointwise matrix product

\[ C_{ij} = \sum_k A_{ik}\,B_{kj} \]

stops being a general multiply–add. It is the count of positions where both factors are one:

\[ C_{ij} = \#{k : A_{ik}=B_{kj}=1} = \operatorname{popcount}!\bigl(\mathrm{row}_i(A)\ \mathrm{AND}\ \mathrm{col}_j(B)\bigr). \]

No scalar multiply in the leaf. Only bitwise AND and a population count along the contraction index \(k\).

mfft-bench now has that leaf on CPU and GPU. This note is the report: how we got there, what the complexity claim actually means, and the numbers on an RTX 5070 Ti.

A year ago I wrote about Matrix Fast Fourier Transform (MFFT): treat matrix entries as polynomials over a digit base, evaluate them at a family of matrix-valued roots of unity that are really just signed permutations, multiply pointwise, and map back. The hope was to shrink the digit-side cost of matmul from something like \(m^2 O(n^\omega)\) toward \(\tilde O(m),O(n^\omega)\).

Theory is cheap. Code is less so. So I built mfft-bench: a C and CUDA benchmark that implements the post’s transform, puts it next to the methods people actually use, and measures both throughput and relative error against a bit-exact product.

This post is the report. It worth mentioning that this post is written by Grok 4.5. Also the code was initiated by Opus 5, but concluded by Grok 4.5.

Did you ever wonder if Quantum mechanics is not following Constructivity's principles, then why it works, at least in the calculation level! I mean the "shut up and calculate" approach of the Copenhagen interpretation worked, right? Notice it's mostly referring to the gaps between the computations that don't have comprehensive explanation rather than the computation of the Quantum mechanics. After all, nobody has problem with the calculation part, since they are constructive and boring!

Anyway, I hope it'll be clear for you in the end of this post why it even works! The Type Mechanics is inspired by Matrix Mechanics of Heisenberg, and also its predecessor the Statistical Mechanics. I would say that this is the third major reversion of this concept. Obviously, there are some differences, which will probably help us to decide how to experimentally validate them.

How do we build our theories? Here is the Philosophical framework to build theories in all scales.

How much can we push for consistently answering deep questions? At least based on my efforts!

Since probability is about counting, you can use it to define our space-time, and more!

Probability is depending on the observer's knowledge, obviously!

Matrix multiplication lies at the heart of modern machine learning, powering everything from neural networks to transformer models. Its optimization can dramatically impact the performance of AI systems, potentially reducing training times from days to hours. Let's explore how we can make this fundamental operation more efficient.

Do you like to have more diverse parties in elections? Let's think about it.

Did you ever come to the point to ask yourself what's the Inner product between a vector and a Bivectors? If you did, you may noticed soon that it doesn't have any meaningful answer yet. I got interested into this problem back when I was studying Mathematical Physics course in my bachelor degree. I wrote my finding to my professor. "wooow! You defined a higher dimensional Levi-Civita-Symbol", he saied. But my invention was much useful than that! This post is the details of that invention. If you know about vectors, and bivectors, etc. I encourage you to read to the end, because this tool is simple and super useful.