Complete Technical Paper Content — Mri Analysis Deep Learning
1. Executive Abstract, Problem Formulation & Background
In statistics and machine learning, ensemble methods use multiple learning algorithms to obtain better predictive performance than could be obtained from any of the constituent learning algorithms alone. Unlike a statistical ensemble in statistical mechanics, which is usually infinite, a machine learning ensemble consists of only a concrete finite set of alternative models, but typically allows for much more flexible structure to exist among those alternatives.
Theoretically, one can justify the diversity concept because the lower bound of the error rate of an ensemble system can be decomposed into accuracy, diversity, and the other term.
The Bayes optimal classifier is a classification technique. It is an ensemble of all the hypotheses in the hypothesis space. On average, no other ensemble can outperform it. The Naive Bayes classifier is a version of this that assumes that the data is conditionally independent on the class and makes the computation more feasible. Each hypothesis is given a vote proportional to the likelihood that the training dataset would be sampled from a system if that hypothesis were true. To facilitate training data of finite size, the vote of each hypothesis is also multiplied by the prior probability of that hypothesis. The Bayes optimal classifier can be expressed with the following equation:
y = a r g m a x c j ∈ C ∑ h i ∈ H P ( c j | h i ) P ( T | h i ) P ( h i ) {\displaystyle y={\underset {c_{j}\in C}{\mathrm {argmax} }}\sum _{h_{i}\in H}{P(c_{j}|h_{i})P(T|h_{i})P(h_{i})}}
where y {\displaystyle y} is the predicted class, C {\displaystyle C} is the set of all possible classes, H {\displaystyle H} is the hypothesis space, P {\displaystyle P} refers to a probability, and T {\displaystyle T} is the training data. As an ensemble, the Bayes optimal classifier represents a hypothesis that is not necessarily in H {\displaystyle H} . The hypothesis represented by the Bayes optimal classifier, however, is the optimal hypothesis in ensemble space (the space of all possible ensembles consisting only of hypotheses in H {\displaystyle H} ). This formula can be restated using Bayes' theorem, which says that the posterior is proportional to the likelihood times the prior:
P ( h i | T ) ∝ P ( T | h i ) P ( h i ) {\displaystyle P(h_{i}|T)\propto P(T|h_{i})P(h_{i})}
2. Related Work, Literature Survey & Algorithmic Foundations
y = a r g m a x c j ∈ C ∑ h i ∈ H P ( c j | h i ) P ( h i | T ) {\displaystyle y={\underset {c_{j}\in C}{\mathrm {argmax} }}\sum _{h_{i}\in H}{P(c_{j}|h_{i})P(h_{i}|T)}}
Boosting involves training successive models by emphasizing training data mis-classified by previously learned models. Initially, all data (D1) has equal weight and is used to learn a base model M1. The examples mis-classified by M1 are assigned a weight greater than correctly classified examples. This boosted data (D2) is used to train a second base model M2, and so on. Inference is done by voting. In some cases, boosting has yielded better accuracy than bagging, but tends to over-fit more. The most common implementation of boosting is Adaboost, but some newer algorithms are reported to achieve better results.
For each model m in the bucket: Do c times: (where 'c' is some constant) Randomly divide the training dataset into two sets: A and B Train m with A Test m with B Select the model that obtains the highest average score
Cross-Validation Selection can be summed up as: "try them all with the training set, and pick the one that works best". Gating is a generalization of Cross-Validation Selection. It involves training another learning model to decide which of the models in the bucket is best-suited to solve the problem. Often, a perceptron is used for the gating model. It can be used to pick the "best" model, or it can be used to give a linear weight to the predictions from each model in the bucket. When a bucket of models is used with a large set of problems, it may be desirable to avoid training some of the models that take a long time to train. Landmark learning is a meta-learning approach that seeks to solve this problem. It involves training only the fast (but imprecise) algorithms in the bucket, and then using the performance of these algorithms to help determine which slow (but accurate) algorithm is most likely to do best.
e k = H ( p , q k ) − λ K ∑ j ≠ k H ( q j , q k ) {\displaystyle e^{k}=H(p,q^{k})-{\frac {\lambda }{K}}\sum _{j\neq k}H(q^{j},q^{k})}
where e k {\displaystyle e^{k}} is the cost function of the k t h {\displaystyle k^{th}} classifier, q k {\displaystyle q^{k}} is the probability of the k t h {\displaystyle k^{th}} classifier, p {\displaystyle p} is the true probability that we need to estimate and λ {\displaystyle \lambda } is a parameter between 0 and 1 that define the diversity that we would like to establish. When λ = 0 {\displaystyle \lambda =0} we want each classifier to do its best regardless of the ensemble and when λ = 1 {\displaystyle \lambda =1} we would like the classifier to be as diverse as possible.
3. Dataset Ingestion, Preprocessing & Stochastic Augmentations
Face recognition, which recently has become one of the most popular research areas of pattern recognition, copes with identification or verification of a person by their digital images. Hierarchical ensembles based on Gabor Fisher classifier and independent component analysis preprocessing techniques are some of the earliest ensembles employed in this field.
While speech recognition is mainly based on deep learning because most of the industry players in this field like Google, Microsoft and IBM reveal that the core technology of their speech recognition is based on this approach, speech-based emotion recognition can also have a satisfactory performance with ensemble learning. It is also being successfully used in facial emotion recognition.
Fraud detection deals with the identification of bank fraud, such as money laundering, credit card fraud and telecommunication fraud, which have vast domains of research and applications of machine learning. Because ensemble learning improves the robustness of the normal behavior modelling, it has been proposed as an efficient technique to detect such fraudulent cases and activities in banking and credit card systems.
Magnetic resonance imaging (MRI) is a medical imaging technique used in radiology to generate pictures of the anatomy and the physiological processes inside the body. MRI scanners use strong magnetic fields, magnetic field gradients, and radio waves to form images of the organs in the body. MRI does not involve X-rays or the use of ionizing radiation, which distinguishes it from computed tomography (CT) and positron emission tomography (PET) scans. MRI is a medical application of nuclear magnetic resonance (NMR), which can also be used for imaging in other NMR applications, such as NMR spectroscopy. MRI is widely used in hospitals and clinics for medical diagnosis, staging and follow-up of disease. In the early 1970s, it was demonstrated by R. Damadian, C.F. Hazlewood and D.C. Chang that measurement of NMR relaxation times can be used to detect cancer. Soon after that, P.C. Lauterbur proposed to use a magnetic field gradient to generate the NMR image in a 2-D plane. Compared to CT, MRI provides better contrast in images of soft tissues, e.g. in the brain or abdomen. However, it may be perceived as less comfortable by patients, due to the usually longer and louder measurements with the subject in a long, confining tube, although "open" MRI designs mostly relieve this. Additionally, implants and other non-removable metal in the body can pose a risk and may exclude some patients from undergoing an MRI examination safely. MRI was originally called NMRI (nuclear magnetic resonance imaging), but "nuclear" was dropped to avoid negative associations. Certain atomic nuclei are able to absorb radio frequency (RF) energy when placed in an external magnetic field; the resultant evolving spin polarization can induce an RF signal in a radio frequency coil and thereby be detected. In other words, the nuclear magnetic spin of protons in the hydrogen nuclei resonates with the RF incident waves and emit coherent radiation with compact direction, energy (frequency) and phase. This coherent amplified radiation is then detected by RF antennas close to the subject being examined. It is a process similar to masers. In clinical and research MRI, hydrogen atoms are most often used to generate a macroscopic polarized radiation that is detected by the antennas. Hydrogen atoms are naturally abundant in humans and other biological organisms, particularly in water and fat. For this reason, most MRI scans essentially map the location of water and fat in the body. Pulses of radio waves excite the nuclear spin energy transition, and magnetic field gradients localize the polarization in space. By varying the parameters of the pulse sequence, different contrasts may be generated between tissues based on the relaxation properties of the hydrogen atoms therein. Since its development in the 1970s and 1980s, MRI has proven to be a versatile imaging technique. While MRI is most prominently used in diagnostic medicine and biomedical research, it also may be used to form images of non-living objects, such as mummies. Diffusion MRI and functional MRI extend the utility of MRI to capture neuronal tracts and blood flow respectively in the nervous system, in addition to detailed spatial images. The sustained increase in demand for MRI within health systems has led to concerns about cost effectiveness and overdiagnosis.
The major components of an MRI scanner are the main magnet, which polarizes the sample, the shim coils for correcting shifts in the homogeneity of the main magnetic field, the gradient system which is used to localize the region to be scanned and the RF system, which excites the sample and detects the resulting NMR signal. The whole system is controlled by one or more computers. In most medical applications, hydrogen nuclei, which consist solely of a proton, that are in tissues create a signal that is processed to form an image of the body in terms of the density of those nuclei in a specific region. Given that the protons are affected by fields from other atoms to which they are bonded, it is possible to separate responses from hydrogen in specific compounds. To perform a study, the person is positioned within an MRI scanner that forms a strong magnetic field around the area to be imaged. First, energy from an oscillating magnetic field is temporarily applied to the patient at the appropriate resonance frequency. Scanning with X and Y gradient coils causes a selected region of the patient to experience the exact magnetic field required for the energy to be absorbed. The atoms are excited by a RF pulse and the resultant signal is measured by one or more receiving coils. The RF signal may be processed to deduce position information by looking at the changes in RF level and phase caused by varying the local magnetic field using gradient coils. As these coils are rapidly switched during the excitation and response to perform a moving line scan, they create the characteristic repetitive noise of an MRI scan as the windings move slightly due to magnetostriction. The contrast between different tissues is determined by the rate at which excited atoms return to the equilibrium state. Exogenous contrast agents may be given to the person to make the image clearer.
MRI requires a magnetic field that is both strong and uniform to a few parts per million across the scan volume. The field strength of the magnet is measured in teslas – and while the majority of systems operate at 1.5 T, commercial systems are available between 0.2 and 7 T. 3T MRI systems, also called 3 Tesla MRIs, have stronger magnets than 1.5 systems and are considered better for images of organs and soft tissue. Whole-body MRI systems for research applications operate in e.g. 9.4T, 10.5T, 11.7T. Even higher field whole-body MRI systems e.g. 14 T and beyond are in conceptual proposal or in engineering design. Most clinical magnets are superconducting magnets, which require liquid helium to keep them at low temperatures. Lower field strengths can be achieved with permanent magnets, which are often used in "open" MRI scanners for claustrophobic patients. Lower field strengths are also used in a portable MRI scanner approved by the FDA in 2020. Recently, MRI has been demonstrated also at ultra-low fields, i.e., in the microtesla-to-millitesla range, where sufficient signal quality is made possible by prepolarization (on the order of 10–100 mT) and by measuring the Larmor precession fields at about 100 microtesla with highly sensitive superconducting quantum interference devices (SQUIDs).
4. Neural Network Backbone, Layer Specifications & Loss Formulations
Each tissue returns to its equilibrium state after excitation by the independent relaxation processes of T1 (spin-lattice; that is, magnetization in the same direction as the static magnetic field) and T2 (spin–spin; transverse to the static magnetic field). To create a T1-weighted image, magnetization is allowed to recover before measuring the MR signal by changing the repetition time (TR). This image weighting is useful for assessing the cerebral cortex, identifying fatty tissue, characterizing focal liver lesions, and in general, obtaining morphological information, as well as for post-contrast imaging.
To create a T2-weighted image, magnetization is allowed to decay before measuring the MR signal by changing the echo time (TE). This image weighting is useful for detecting edema and inflammation, revealing white matter lesions, and assessing zonal anatomy in the prostate and uterus. The information from MRI scans comes in the form of image contrasts based on differences in the rate of relaxation of nuclear spins following their perturbation by an oscillating magnetic field (in the form of radiofrequency pulses through the sample). The relaxation rates are a measure of the time it takes for a signal to decay back to an equilibrium state from either the longitudinal or transverse plane. Magnetization builds up along the z-axis in the presence of a magnetic field, B0, such that the magnetic dipoles in the sample will, on average, align with the z-axis summing to a total magnetization Mz. This magnetization along z is defined as the equilibrium magnetization; magnetization is defined as the sum of all magnetic dipoles in a sample. Following the equilibrium magnetization, a 90° radiofrequency (RF) pulse flips the direction of the magnetization vector in the xy-plane, and is then switched off. The initial magnetic field B0, however, is still applied. Thus, the spin magnetization vector will slowly return from the xy-plane back to the equilibrium state. The time it takes for the magnetization vector to return to its equilibrium value, Mz, is referred to as the longitudinal relaxation time, T1. Subsequently, the rate at which this happens is simply the reciprocal of the relaxation time: 1 T 1 = R 1 {\displaystyle {\frac {1}{T_{1}}}=R_{1}} . Similarly, the time in which it takes for Mxy to return to zero is T2, with the rate 1 T 2 = R 2 {\displaystyle {\frac {1}{T_{2}}}=R_{2}} . Magnetization as a function of time is defined by the Bloch equations. T1 and T2 values are dependent on the chemical environment of the sample; hence their utility in MRI. Soft tissue and muscle tissue relax at different rates, yielding the image contrast in a typical scan. The standard display of MR images is to represent fluid characteristics in black-and-white images, where different tissues turn out as follows:
MRI has a wide range of applications in medical diagnosis and around 50,000 scanners are estimated to be in use worldwide. MRI affects diagnosis and treatment in many specialties although the effect on improved health outcomes is disputed in certain cases. MRI is the investigation of choice in the preoperative staging of rectal and prostate cancer and has a role in the diagnosis, staging, and follow-up of other tumors, as well as for determining areas of tissue for sampling in biobanking.
MRI is the investigative tool of choice for neurological cancers over CT, as it offers better visualization of the posterior cranial fossa, containing the brainstem and the cerebellum. The contrast provided between grey and white matter makes MRI the best choice for many conditions of the central nervous system, including demyelinating diseases, dementia, cerebrovascular disease, infectious diseases, Alzheimer's disease and epilepsy. Since multiple images are taken milliseconds apart, it can show how the brain responds to different stimuli, enabling researchers to study both functional and structural brain abnormalities in psychological disorders. MRI also is used in guided stereotactic surgery and radiosurgery for treatment of intracranial tumors, arteriovenous malformations, and other surgically treatable conditions using a device known as the N-localizer. New tools that implement artificial intelligence in healthcare have demonstrated higher image quality and morphometric analysis in neuroimaging with the application of a denoising system. More broadly, deep learning has been applied to accelerate MRI acquisition itself by reconstructing diagnostic-quality images from undersampled k-space data. The fastMRI project, a collaboration between Meta AI Research (FAIR) and NYU Langone Health launched in 2018, released the largest open-source dataset of raw MRI measurements and demonstrated that AI-based reconstruction can achieve up to 4× acceleration of knee and brain scans with no loss of diagnostic accuracy—a result confirmed in a clinical interchangeability study. These methods have since been adopted as a clinical standard for accelerated MRI at several major medical centers. The record for the highest spatial resolution of a whole intact brain (postmortem) is 100 microns, from Massachusetts General Hospital. The data was published in Nature in October 2019. Though MRI is used widely in research on mental disabilities, based on a 2024 systematic literature review and meta analysis commissioned by the Patient-Centered Outcomes Research Institute (PCORI), available research using MRI scans to diagnose ADHD showed great variability. The authors conclude that MRI cannot be reliably used to assist in making a clinical diagnosis of ADHD.
Cardiac MRI is complementary to other imaging techniques, such as echocardiography, cardiac CT, and nuclear medicine. It can be used to assess the structure and the function of the heart. Its applications include assessment of myocardial ischemia and viability, cardiomyopathies, myocarditis, iron overload, vascular diseases, and congenital heart disease.
Applications in the musculoskeletal system include spinal imaging, assessment of joint disease, and soft tissue tumors. MRI techniques can also be used for diagnostic imaging of systemic muscle diseases including genetic muscle diseases. Swallowing movements of the throat and esophagus can cause motion artifacts over the imaged spine. Therefore, a saturation pulse applied over this region can help to avoid these artifacts. Motion artifacts arising due to the pumping of the heart can be reduced by timing the MRI pulse according to heart cycles. Blood vessel flow artifacts can be reduced by applying saturation pulses above and below the region of interest.
5. Training Protocol, Convergence Behavior & Experimental Evaluation
Magnetic resonance angiography (MRA) generates pictures of the arteries to evaluate them for stenosis (abnormal narrowing) or aneurysms (vessel wall dilatations, at risk of rupture). MRA is often used to evaluate the arteries of the neck and brain, the thoracic and abdominal aorta, the renal arteries, and the legs (called a "run-off"). A variety of techniques can be used to generate the pictures, such as administration of a paramagnetic contrast agent (gadolinium) or using a technique known as "flow-related enhancement" (e.g., 2D and 3D time-of-flight sequences), where most of the signal on an image is due to blood that recently moved into that plane (see also FLASH MRI). Techniques involving phase accumulation (known as phase contrast angiography) can also be used to generate flow velocity maps easily and accurately. Magnetic resonance venography (MRV) is a similar procedure that is used to image veins. In this method, the tissue is now excited inferiorly, while the signal is gathered in the plane immediately superior to the excitation plane—thus imaging the venous blood that recently moved from the excited plane.
MRI for imaging anatomical structures or blood flow do not require contrast agents since the varying properties of the tissues or blood provide natural contrasts. However, for more specific types of imaging, exogenous contrast agents may be given intravenously, orally, or intra-articularly. Most contrast agents are either paramagnetic (e.g.: gadolinium, manganese, europium), and are used to shorten T1 in the tissue they accumulate in, or super-paramagnetic (SPIONs), and are used to shorten T2 and T2* in healthy tissue reducing its signal intensity (negative contrast agents). The most commonly used intravenous contrast agents are based on chelates of gadolinium, which is highly paramagnetic. In general, these agents have proved safer than the iodinated contrast agents used in X-ray radiography or CT. Anaphylactoid reactions are rare, occurring in approx. 0.03–0.1%. Of particular interest is the lower incidence of nephrotoxicity, compared with iodinated agents, when given at usual doses—this has made contrast-enhanced MRI scanning an option for patients with renal impairment, who would otherwise not be able to undergo contrast-enhanced CT. Gadolinium-based contrast reagents are typically octadentate complexes of gadolinium(III). The complex is very stable (log K > 20) so that, in use, the concentration of the un-complexed Gd3+ ions should be below the toxicity limit. The 9th place in the metal ion's coordination sphere is occupied by a water molecule which exchanges rapidly with water molecules in the reagent molecule's immediate environment, affecting the magnetic resonance relaxation time. In December 2017, the Food and Drug Administration (FDA) in the United States announced in a drug safety communication that new warnings were to be included on all gadolinium-based contrast agents (GBCAs). The FDA also called for increased patient education and requiring gadolinium contrast vendors to conduct additional animal and clinical studies to assess the safety of these agents. Although gadolinium agents have proved useful for patients with kidney impairment, in patients with severe kidney failure requiring dialysis there is a risk of a rare but serious illness, nephrogenic systemic fibrosis, which may be linked to the use of certain gadolinium-containing agents. The most frequently linked is gadodiamide, but other agents have been linked too. Although a causal link has not been definitively established, current guidelines in the United States are that dialysis patients should only receive gadolinium agents where essential and that dialysis should be performed as soon as possible after the scan to remove the agent from the body promptly. In Europe, where more gadolinium-containing agents are available, a classification of agents according to potential risks has been released. In 2008, a new contrast agent named gadoxetate, brand name Eovist (US) or Primovist (EU), was approved for diagnostic use: This has the theoretical benefit of a dual excretion path.
An MRI sequence is a particular setting of radiofrequency pulses and gradients, resulting in a particular image appearance. The T1 and T2 weighting can also be described as MRI sequences.
Magnetic resonance spectroscopy (MRS) is used to measure the levels of different metabolites in body tissues, which can be achieved through a variety of single voxel or imaging-based techniques. The MR signal produces a spectrum of resonances that corresponds to different molecular arrangements of the isotope being "excited". This signature is used to diagnose certain metabolic disorders, especially those affecting the brain, and to provide information on tumor metabolism. Magnetic resonance spectroscopic imaging (MRSI) combines both spectroscopic and imaging methods to produce spatially localized spectra from within the sample or patient. The spatial resolution is much lower (limited by the available SNR), but the spectra in each voxel contains information about many metabolites. Because the available signal is used to encode spatial and spectral information, MRSI requires high SNR achievable only at higher field strengths (3 T and above). The high procurement and maintenance costs of MRI with extremely high field strengths inhibit their popularity. However, recent compressed sensing-based software algorithms (e.g., SAMV) have been proposed to achieve super-resolution without requiring such high field strengths.
The lack of harmful effects on the patient and the operator make MRI well-suited for interventional radiology, where the images produced by an MRI scanner guide minimally invasive procedures. Such procedures use no ferromagnetic instruments. A specialized growing subset of interventional MRI is intraoperative MRI, in which an MRI is used in surgery. Some specialized MRI systems allow imaging concurrent with the surgical procedure. More typically, the surgical procedure is temporarily interrupted so that MRI can assess the success of the procedure or guide subsequent surgical work.
Hydrogen has the most frequently imaged nucleus in MRI because it is present in biological tissues in great abundance, and because its high gyromagnetic ratio gives a strong signal. However, any nucleus with a net nuclear spin could potentially be imaged with MRI. Such nuclei include deuterium, helium-3, lithium-7, carbon-13, fluorine-19, oxygen-17, sodium-23, phosphorus-31 and xenon-129. 2H, 23Na and 31P are naturally abundant in the body, so they can be imaged directly. Naturally abundant deuterium at the concentration of around 15mM can be imaged, but suffers from low gamma sensitivity and quadripolar Relaxation (NMR). Deuterium imaging however has a sparse chemical shift spectrum making it possible to develop tailored multiband selective RF pulses for metabolite selective imaging. Thus, metabolic imaging, similar to what's done with Carbon-13 is possible with Deuterium metabolic imaging (DMI) for insights into vivo metabolic processes. As well, the short T2 of deuterium allows it to be signal averaged rapidly, making up for some of its physical shortcomings. Gaseous isotopes such as 3He or 129Xe must be hyperpolarized and then inhaled as their nuclear density is too low to yield a useful signal under normal conditions. 17O and 19F can be administered in sufficient quantities in liquid form (e.g. 17O-water) that hyperpolarization is not a necessity. Using helium or xenon has the advantage of reduced background noise, and therefore increased contrast for the image itself, because these elements are not normally present in biological tissues. Moreover, the nucleus of any atom that has a net nuclear spin and that is bonded to a hydrogen atom could potentially be imaged via heteronuclear magnetization transfer MRI that would image the high-gyromagnetic-ratio hydrogen nucleus instead of the low-gyromagnetic-ratio nucleus that is bonded to the hydrogen atom. In principle, heteronuclear magnetization transfer MRI could be used to detect the presence or absence of specific chemical bonds. Multinuclear imaging is primarily a research technique at present. However, potential applications include functional imaging and imaging of organs poorly seen on 1H MRI (e.g., lungs and bones) or as alternative contrast agents. Inhaled hyperpolarized 3He can be used to image the distribution of air spaces within the lungs. Injectable solutions containing 13C or stabilized bubbles of hyperpolarized 129Xe have been studied as contrast agents for angiography and perfusion imaging. 31P can potentially provide information on bone density and structure, as well as functional imaging of the brain. Multinuclear imaging holds the potential to chart the distribution of lithium in the human brain, this element finding use as an important drug for those with conditions such as bipolar disorder.
Model Figures & Evaluation Curves (4 Images)
Deep Learning Execution Pipeline
(Preprocessing)
(Feature Extraction)
(Self-Attention)
(Loss Optimization)