Complete Technical Paper Content — Galaxy Classification Ai
1. Executive Abstract, Problem Formulation & Background
Max Tegmark (born 5 May 1967) is a Swedish-American physicist, machine learning researcher and author. He is a professor of physics at the Massachusetts Institute of Technology. He co-founded and leads the Future of Life Institute, a nonprofit focused on reducing global catastrophic risks from advanced technologies. Originally a cosmologist, Tegmark's focus shifted toward artificial intelligence (AI). His 2017 book Life 3.0 presents scenarios for what the world might look like as AI continues to develop. Tegmark advocates for a halt on the development of artificial superintelligence.
Under Tegmark's founding leadership, the Future of Life Institute has pursued a stated mission to "steer transformative technologies away from extreme, large-scale risks and towards benefiting life". It is focused on research aiming to mitigate large-scale risks for humanity, particularly those related to advanced AI. A co-founding faculty member was University of California, Santa Cruz professor Anthony Aguirre, now Executive Director of the organization. Its board-level leadership has included Elon Musk, Skype and Kazaa founder Jaan Tallinn, as well as actors (Alan Alda and Morgan Freeman), and individual graduate students (including his wife, Meia Chita-Tegmark, then a Boston University PhD student). Tegmark and the organization are academic proponents of approaches and views that are aware of and wrestle with the potential risks associated with the development of AI. It received a cryptocurrency donation of $665 million donation from Vitalik Buterin in 2021, as well as a $10 million donation from Elon Musk in 2015 and additional funding by board member and Skype co-founder Jaan Tallinn. In 2023, Tegmark was the focus of a controversy when he was alleged to have signed a letter of intent on behalf of the Future of Life Institute for a $100,000 grant—ultimately rejected—to far-right media outlet Nya Dagbladet, an outlet for which Tegmark's brother wrote, an allegation to which the Institute formally responded. Tegmark later said that the Institute "ultimately decided to reject it because of what our subsequent due diligence uncovered", that they rejected it long before the media became involved, and that the institute "finds Nazi, neo-Nazi or pro-Nazi groups or ideologies despicable and would never knowingly support them". An official statement from the Future of Life Institute further expands on this: "FLI finds groups or ideologies espousing antisemitism, white supremacy, or racism despicable and would never knowingly support any such group".
A generative adversarial network (GAN) is a class of machine learning frameworks and a prominent framework for approaching generative AI. The concept was initially developed by Ian Goodfellow and his colleagues at Université de Montréal in June 2014. In a GAN, two neural networks compete with each other in the form of a zero-sum game, where one agent's gain is another agent's loss. Given a training set, this technique learns to generate new data with the same statistics as the training set. For example, a GAN trained on photographs can generate new photographs that look at least superficially authentic to human observers, having many realistic characteristics. Though originally proposed as a form of generative model for unsupervised learning, GANs have also proved useful for semi-supervised learning, fully supervised learning, and reinforcement learning. The core idea of a GAN is based on the "indirect" training through the discriminator, another neural network that can tell how "realistic" the input seems, which itself is also being updated dynamically. This means that the generator is not trained to minimize the distance to a specific image, but rather to fool the discriminator. This enables the model to learn in an unsupervised manner. GANs are similar to mimicry in evolutionary biology, with an evolutionary arms race between both networks.
The generator aims to minimize the objective, and the discriminator aims to maximize the objective.
The generator's task is to approach μ G ≈ μ ref {\displaystyle \mu _{G}\approx \mu _{\text{ref}}} , that is, to match its own output distribution as closely as possible to the reference distribution. The discriminator's task is to output a value close to 1 when the input appears to be from the reference distribution, and to output a value close to 0 when the input looks like it came from the generator distribution.
Compared to fully visible belief networks such as WaveNet and PixelRNN and autoregressive models in general, GANs can generate one complete sample in one pass, rather than multiple passes through the network. Compared to Boltzmann machines and linear ICA, there is no restriction on the type of function used by the network. Since neural networks are universal approximators, GANs are asymptotically consistent. As of 2026, variational autoencoders have been proven to be universal approximators.
In modern probability theory based on measure theory, a probability space also needs to be equipped with a σ-algebra. As a result, a more rigorous definition of the GAN game would make the following changes:Each probability space ( Ω , B , μ ref ) {\displaystyle (\Omega ,{\mathcal {B}},\mu _{\text{ref}})} defines a GAN game. The generator's strategy set is P ( Ω , B ) {\displaystyle {\mathcal {P}}(\Omega ,{\mathcal {B}})} , the set of all probability measures μ G {\displaystyle \mu _{G}} on the measure-space ( Ω , B ) {\displaystyle (\Omega ,{\mathcal {B}})} .
2. Related Work, Literature Survey & Algorithmic Foundations
The discriminator's strategy set is the set of Markov kernels μ D : ( Ω , B ) → P ( [ 0 , 1 ] , B ( [ 0 , 1 ] ) ) {\displaystyle \mu _{D}:(\Omega ,{\mathcal {B}})\to {\mathcal {P}}([0,1],{\mathcal {B}}([0,1]))} , where B ( [ 0 , 1 ] ) {\displaystyle {\mathcal {B}}([0,1])} is the Borel σ-algebra on [ 0 , 1 ] {\displaystyle [0,1]} .Since issues of measurability never arise in practice, these will not concern us further.
If both the generator's and the discriminator's strategy sets are spanned by a finite number of strategies, then by the minimax theorem, min μ G max μ D L ( μ G , μ D ) = max μ D min μ G L ( μ G , μ D ) {\displaystyle \min _{\mu _{G}}\max _{\mu _{D}}L(\mu _{G},\mu _{D})=\max _{\mu _{D}}\min _{\mu _{G}}L(\mu _{G},\mu _{D})} that is, the move order does not matter. However, since the strategy sets are both not finitely spanned, the minimax theorem does not apply, and the idea of an "equilibrium" becomes delicate. To wit, there are the following different concepts of equilibrium:
Equilibrium when generator moves first, and discriminator moves second: μ ^ G ∈ arg min μ G max μ D L ( μ G , μ D ) , μ ^ D ∈ arg max μ D L ( μ ^ G , μ D ) , {\displaystyle {\hat {\mu }}_{G}\in \arg \min _{\mu _{G}}\max _{\mu _{D}}L(\mu _{G},\mu _{D}),\quad {\hat {\mu }}_{D}\in \arg \max _{\mu _{D}}L({\hat {\mu }}_{G},\mu _{D}),\quad }
Equilibrium when discriminator moves first, and generator moves second: μ ^ D ∈ arg max μ D min μ G L ( μ G , μ D ) , μ ^ G ∈ arg min μ G L ( μ G , μ ^ D ) , {\displaystyle {\hat {\mu }}_{D}\in \arg \max _{\mu _{D}}\min _{\mu _{G}}L(\mu _{G},\mu _{D}),\quad {\hat {\mu }}_{G}\in \arg \min _{\mu _{G}}L(\mu _{G},{\hat {\mu }}_{D}),}
Nash equilibrium ( μ ^ D , μ ^ G ) {\displaystyle ({\hat {\mu }}_{D},{\hat {\mu }}_{G})} , which is stable under simultaneous move order: μ ^ D ∈ arg max μ D L ( μ ^ G , μ D ) , μ ^ G ∈ arg min μ G L ( μ G , μ ^ D ) {\displaystyle {\hat {\mu }}_{D}\in \arg \max _{\mu _{D}}L({\hat {\mu }}_{G},\mu _{D}),\quad {\hat {\mu }}_{G}\in \arg \min _{\mu _{G}}L(\mu _{G},{\hat {\mu }}_{D})}
For general games, these equilibria do not have to agree, or even to exist. For the original GAN game, these equilibria all exist, and are all equal. However, for more general GAN games, these do not necessarily exist, or agree.
D ∗ ( x ) 1 − D ∗ ( x ) = ρ ref ( x ) ρ G ( x ) μ -almost everywhere on { ρ G > 0 } . {\displaystyle {\frac {D^{*}(x)}{1-D^{*}(x)}}={\frac {\rho _{\text{ref}}(x)}{\rho _{G}(x)}}\quad \mu {\text{-almost everywhere on }}\{\rho _{G}>0\}.}
3. Dataset Ingestion, Preprocessing & Stochastic Augmentations
On { ρ G = 0 } {\displaystyle \{\rho _{G}=0\}} , D ∗ = 1 {\displaystyle D^{*}=1} ; on { ρ ref = 0 } {\displaystyle \{\rho _{\text{ref}}=0\}} , D ∗ = 0 {\displaystyle D^{*}=0} , in each case μ {\displaystyle \mu } -almost everywhere. If μ ref ≪ μ G {\displaystyle \mu _{\text{ref}}\ll \mu _{G}} , the displayed ratio is a version of d μ ref / d μ G {\displaystyle d\mu _{\text{ref}}/d\mu _{G}} μ G {\displaystyle \mu _{G}} -almost everywhere. Where both densities are positive,
D ∗ ( x ) = σ ( ln ρ ref ( x ) − ln ρ G ( x ) ) μ -almost everywhere , {\displaystyle D^{*}(x)=\sigma \left(\ln \rho _{\text{ref}}(x)-\ln \rho _{G}(x)\right)\quad \mu {\text{-almost everywhere}},}
where σ {\displaystyle \sigma } is the logistic function. In the equal-prior mixture model M = ( μ ref + μ G ) / 2 {\displaystyle M=(\mu _{\text{ref}}+\mu _{G})/2} , D ∗ {\displaystyle D^{*}} is a version of the posterior probability that an observation came from the reference distribution, M {\displaystyle M} -almost everywhere.
GANs often suffer from mode collapse where they fail to generalize properly, missing entire modes from the input data. For example, a GAN trained on the MNIST dataset containing many samples of each digit might only generate pictures of digit 0. This was termed "the Helvetica scenario". A typical mechanism for mode collapse is the generator only generating one or a few of the likely values, or a very incomplete picture of the target distribution. As the discriminator is only trained to distinguish real from fake samples, it will correctly identify the generated samples as real, but no penalty is imposed on the GAN's ability to generate data that represents the full range of the target distribution. Weak discriminators, for instance underparametrized ones, or ones trained too slow compared to the generator, may as well be unable to fully discriminate over the entire support of the distribution, and only become able to correctly discriminate a very incomplete part of the target distribution. Some researchers perceive the root problem to be a weak discriminative network that fails to notice the pattern of omission, while others assign blame to a bad choice of objective function. Many solutions have been proposed, but it is still an open problem. Even the state-of-the-art architecture, BigGAN (2019), could not avoid mode collapse. The authors resorted to "allowing collapse to occur at the later stages of training, by which time a model is sufficiently trained to achieve good results".
Original GAN, non-saturating loss: This objective for generator was recommended in the original paper for faster convergence. L G = E x ∼ μ G [ ln D ( x ) ] {\displaystyle L_{G}=\operatorname {E} _{x\sim \mu _{G}}[\ln D(x)]} The effect of using this objective is analyzed in Section 2.2.2 of Arjovsky et al. Original GAN, maximum likelihood:
L G = E x ∼ μ G [ ( exp ∘ σ − 1 ∘ D ) ( x ) ] {\displaystyle L_{G}=\operatorname {E} _{x\sim \mu _{G}}[({\exp }\circ \sigma ^{-1}\circ D)(x)]} where σ {\displaystyle \sigma } is the logistic function. When the discriminator is optimal, the generator gradient is the same as in maximum likelihood estimation, even though GAN cannot perform maximum likelihood estimation itself. Hinge loss GAN: L D = − E x ∼ p ref [ min ( 0 , − 1 + D ( x ) ) ] − E x ∼ μ G [ min ( 0 , − 1 − D ( x ) ) ] {\displaystyle L_{D}=-\operatorname {E} _{x\sim p_{\text{ref}}}\left[\min \left(0,-1+D(x)\right)\right]-\operatorname {E} _{x\sim \mu _{G}}\left[\min \left(0,-1-D\left(x\right)\right)\right]}
L G = − E x ∼ μ G [ D ( x ) ] {\displaystyle L_{G}=-\operatorname {E} _{x\sim \mu _{G}}[D(x)]} Least squares GAN: L D = E x ∼ μ ref [ ( D ( x ) − b ) 2 ] + E x ∼ μ G [ ( D ( x ) − a ) 2 ] {\displaystyle L_{D}=\operatorname {E} _{x\sim \mu _{\text{ref}}}[(D(x)-b)^{2}]+\operatorname {E} _{x\sim \mu _{G}}[(D(x)-a)^{2}]}
4. Neural Network Backbone, Layer Specifications & Loss Formulations
L G = E x ∼ μ G [ ( D ( x ) − c ) 2 ] {\displaystyle L_{G}=\operatorname {E} _{x\sim \mu _{G}}[(D(x)-c)^{2}]} where a , b , c {\displaystyle a,b,c} are parameters to be chosen. The authors recommended a = − 1 , b = 1 , c = 0 {\displaystyle a=-1,b=1,c=0} .
The discriminator's strategy set is the set of measurable functions of type D : Ω → R {\displaystyle D:\Omega \to \mathbb {R} } with bounded Lipschitz norm: ‖ D ‖ L ≤ K {\displaystyle \|D\|_{L}\leq K} , where K {\displaystyle K} is a fixed positive constant. The objective is L W G A N ( μ G , D ) := E x ∼ μ G [ D ( x ) ] − E x ∼ μ ref [ D ( x ) ] {\displaystyle L_{WGAN}(\mu _{G},D):=\operatorname {E} _{x\sim \mu _{G}}[D(x)]-\mathbb {E} _{x\sim \mu _{\text{ref}}}[D(x)]}
One of its purposes is to solve the problem of mode collapse (see above). The authors claim "In no experiment did we see evidence of mode collapse for the WGAN algorithm".
( Ω X , μ ref ) {\displaystyle (\Omega _{X},\mu _{\text{ref}})} , the space of reference images.
( Ω Z , μ Z ) {\displaystyle (\Omega _{Z},\mu _{Z})} , the fixed random noise generator.
( Ω C , μ C ) {\displaystyle (\Omega _{C},\mu _{C})} , the fixed random information generator. There are 3 players in 2 teams: generator, Q, and discriminator. The generator and Q are on one team, and the discriminator on the other team. The objective function is L ( G , Q , D ) = L G A N ( G , D ) − λ I ^ ( G , Q ) {\displaystyle L(G,Q,D)=L_{GAN}(G,D)-\lambda {\hat {I}}(G,Q)} where L G A N ( G , D ) = E x ∼ μ ref , [ ln D ( x ) ] + E z ∼ μ Z [ ln ( 1 − D ( G ( z , c ) ) ) ] {\displaystyle L_{GAN}(G,D)=\operatorname {E} _{x\sim \mu _{\text{ref}},}[\ln D(x)]+\operatorname {E} _{z\sim \mu _{Z}}[\ln(1-D(G(z,c)))]} is the original GAN game objective, and I ^ ( G , Q ) = E z ∼ μ Z , c ∼ μ C [ ln Q ( c ∣ G ( z , c ) ) ] {\displaystyle {\hat {I}}(G,Q)=\mathbb {E} _{z\sim \mu _{Z},c\sim \mu _{C}}[\ln Q(c\mid G(z,c))]}
Generator-Q team aims to minimize the objective, and discriminator aims to maximize it: min G , Q max D L ( G , Q , D ) {\displaystyle \min _{G,Q}\max _{D}L(G,Q,D)}
5. Training Protocol, Convergence Behavior & Experimental Evaluation
( Ω X , μ X ) {\displaystyle (\Omega _{X},\mu _{X})} , the space of reference images.
( Ω Z , μ Z ) {\displaystyle (\Omega _{Z},\mu _{Z})} , the latent space. There are 3 players in 2 teams: generator, encoder, and discriminator. The generator and encoder are on one team, and the discriminator on the other team. The generator's strategies are functions G : Ω Z → Ω X {\displaystyle G:\Omega _{Z}\to \Omega _{X}} , and the encoder's strategies are functions E : Ω X → Ω Z {\displaystyle E:\Omega _{X}\to \Omega _{Z}} . The discriminator's strategies are functions D : Ω X → [ 0 , 1 ] {\displaystyle D:\Omega _{X}\to [0,1]} . The objective function is L ( G , E , D ) = E x ∼ μ X [ ln D ( x , E ( x ) ) ] + E z ∼ μ Z [ ln ( 1 − D ( G ( z ) , z ) ) ] {\displaystyle L(G,E,D)=\mathbb {E} _{x\sim \mu _{X}}[\ln D(x,E(x))]+\mathbb {E} _{z\sim \mu _{Z}}[\ln(1-D(G(z),z))]}
Generator-encoder team aims to minimize the objective, and discriminator aims to maximize it: min G , E max D L ( G , E , D ) {\displaystyle \min _{G,E}\max _{D}L(G,E,D)} In the paper, they gave a more abstract definition of the objective as: L ( G , E , D ) = E ( x , z ) ∼ μ E , X [ ln D ( x , z ) ] + E ( x , z ) ∼ μ G , Z [ ln ( 1 − D ( x , z ) ) ] {\displaystyle L(G,E,D)=\mathbb {E} _{(x,z)\sim \mu _{E,X}}[\ln D(x,z)]+\mathbb {E} _{(x,z)\sim \mu _{G,Z}}[\ln(1-D(x,z))]} where μ E , X ( d x , d z ) = μ X ( d x ) ⋅ δ E ( x ) ( d z ) {\displaystyle \mu _{E,X}(dx,dz)=\mu _{X}(dx)\cdot \delta _{E(x)}(dz)} is the probability distribution on Ω X × Ω Z {\displaystyle \Omega _{X}\times \Omega _{Z}} obtained by pushing μ X {\displaystyle \mu _{X}} forward via x ↦ ( x , E ( x ) ) {\displaystyle x\mapsto (x,E(x))} , and μ G , Z ( d x , d z ) = δ G ( z ) ( d x ) ⋅ μ Z ( d z ) {\displaystyle \mu _{G,Z}(dx,dz)=\delta _{G(z)}(dx)\cdot \mu _{Z}(dz)} is the probability distribution on Ω X × Ω Z {\displaystyle \Omega _{X}\times \Omega _{Z}} obtained by pushing μ Z {\displaystyle \mu _{Z}} forward via z ↦ ( G ( x ) , z ) {\displaystyle z\mapsto (G(x),z)} . Applications of bidirectional models include semi-supervised learning, interpretable machine learning, and neural machine translation.
The CycleGAN game is defined as follows:There are two probability spaces ( Ω X , μ X ) , ( Ω Y , μ Y ) {\displaystyle (\Omega _{X},\mu _{X}),(\Omega _{Y},\mu _{Y})} , corresponding to the two domains needed for translations fore-and-back. There are 4 players in 2 teams: generators G X : Ω X → Ω Y , G Y : Ω Y → Ω X {\displaystyle G_{X}:\Omega _{X}\to \Omega _{Y},G_{Y}:\Omega _{Y}\to \Omega _{X}} , and discriminators D X : Ω X → [ 0 , 1 ] , D Y : Ω Y → [ 0 , 1 ] {\displaystyle D_{X}:\Omega _{X}\to [0,1],D_{Y}:\Omega _{Y}\to [0,1]} . The objective function is L ( G X , G Y , D X , D Y ) = L G A N ( G X , D X ) + L G A N ( G Y , D Y ) + λ L c y c l e ( G X , G Y ) {\displaystyle L(G_{X},G_{Y},D_{X},D_{Y})=L_{GAN}(G_{X},D_{X})+L_{GAN}(G_{Y},D_{Y})+\lambda L_{cycle}(G_{X},G_{Y})}
where λ {\displaystyle \lambda } is a positive adjustable parameter, L G A N {\displaystyle L_{GAN}} is the GAN game objective, and L c y c l e {\displaystyle L_{cycle}} is the cycle consistency loss: L c y c l e ( G X , G Y ) = E x ∼ μ X ‖ G X ( G Y ( x ) ) − x ‖ + E y ∼ μ Y ‖ G Y ( G X ( y ) ) − y ‖ {\displaystyle L_{cycle}(G_{X},G_{Y})=E_{x\sim \mu _{X}}\|G_{X}(G_{Y}(x))-x\|+E_{y\sim \mu _{Y}}\|G_{Y}(G_{X}(y))-y\|} The generators aim to minimize the objective, and the discriminators aim to maximize it: min G X , G Y max D X , D Y L ( G X , G Y , D X , D Y ) {\displaystyle \min _{G_{X},G_{Y}}\max _{D_{X},D_{Y}}L(G_{X},G_{Y},D_{X},D_{Y})} Unlike previous work like pix2pix, which requires paired training data, cycleGAN requires no paired data. For example, to train a pix2pix model to turn a summer scenery photo to winter scenery photo and back, the dataset must contain pairs of the same place in summer and winter, shot at the same angle; cycleGAN would only need a set of summer scenery photos, and an unrelated set of winter scenery photos.
Just before, the GAN game consists of the pair G N , D N {\displaystyle G_{N},D_{N}} generating and discriminating 4x4 images. Just after, the GAN game consists of the pair ( ( 1 − α ) + α ⋅ G N − 1 ) ∘ u ∘ G N , D N ∘ d ∘ ( ( 1 − α ) + α ⋅ D N − 1 ) {\displaystyle ((1-\alpha )+\alpha \cdot G_{N-1})\circ u\circ G_{N},D_{N}\circ d\circ ((1-\alpha )+\alpha \cdot D_{N-1})} generating and discriminating 8x8 images. Here, the functions u , d {\displaystyle u,d} are image up- and down-sampling functions, and α {\displaystyle \alpha } is a blend-in factor (much like an alpha in image composing) that smoothly glides from 0 to 1.
StyleGAN-1 is designed as a combination of Progressive GAN with neural style transfer. The key architectural choice of StyleGAN-1 is a progressive growth mechanism, similar to Progressive GAN. Each generated image starts as a constant 4 × 4 × 512 {\displaystyle 4\times 4\times 512} array, and repeatedly passed through style blocks. Each style block applies a "style latent vector" via affine transform ("adaptive instance normalization"), similar to how neural style transfer uses Gramian matrix. It then adds noise, and normalize (subtract the mean, then divide by the variance). At training time, usually only one style latent vector is used per image generated, but sometimes two ("mixing regularization") in order to encourage each style block to independently perform its stylization without expecting help from other style blocks (since they might receive an entirely different style latent vector). After training, multiple style latent vectors can be fed into each style block. Those fed to the lower layers control the large-scale styles, and those fed to the higher layers control the fine-detail styles. Style-mixing between two images x , x ′ {\displaystyle x,x'} can be performed as well. First, run a gradient descent to find z , z ′ {\displaystyle z,z'} such that G ( z ) ≈ x , G ( z ′ ) ≈ x ′ {\displaystyle G(z)\approx x,G(z')\approx x'} . This is called "projecting an image back to style latent space". Then, z {\displaystyle z} can be fed to the lower style blocks, and z ′ {\displaystyle z'} to the higher style blocks, to generate a composite image that has the large-scale style of x {\displaystyle x} , and the fine-detail style of x ′ {\displaystyle x'} . Multiple images can also be composed this way.
Model Figures & Evaluation Curves (4 Images)
Deep Learning Execution Pipeline
(Preprocessing)
(Feature Extraction)
(Self-Attention)
(Loss Optimization)