Ai Match Outcome Prediction | AI Final Year Project Report

Complete Technical Paper Content — Ai Match Outcome Prediction

1. Executive Abstract, Problem Formulation & Background

The history of artificial intelligence (AI) began in antiquity, with myths, stories, and rumors of artificial beings endowed with intelligence by master craftsmen. The field of research that eventually evolved into what is widely described as artificial intelligence was founded at a workshop held on the campus of Dartmouth College in 1956. At the workshop, a concept of the first AI program, Logic Theorist, was presented by future Turing Awardee Allen Newell and future Nobel Laureate Herbert A. Simon, with help from J. C. Shaw. The software architecture of this program differed from modern AI systems in that it was a hand-written fixed script. Attendees of the workshop became the leaders of AI research for decades. Many of them predicted that machines as intelligent as humans would exist within a generation. Despite the eventual success of its attendees and the field as a whole, the workshop was considered a failure. "When the workshop failed to deliver, AI was dismissed as a pipe dream and research funding dried up". It became apparent that researchers had underestimated the complexity of this objective. In 1974, criticism from James Lighthill and pressure from the U.S. Congress led the U.S. and British Governments to stop funding undirected research into artificial intelligence. Seven years later, a visionary initiative by the Japanese Government and the success of expert systems reinvigorated investment in AI, and by the late 1980s, the industry had grown into a billion-dollar enterprise. However, investors' enthusiasm waned in the 1990s, and the field was criticized in the press and avoided by industry (a period known as an "AI winter"). Nevertheless, research and funding continued to grow under other names. In the early 2000s, machine learning was applied to a wide range of problems in academia and industry. The success was attributable to the availability of powerful computer hardware, the accumulation of expansive data sets, and the application of rigorous mathematical methods. Soon after, deep learning proved to be a breakthrough technology, eclipsing all other methods. The transformer architecture, introduced in 2017, and was utilized to produce generative AI applications, among other implementation. Investment in AI boomed in the 2020s. The AI boom, initiated by the development of transformer architecture, led to the rapid scaling and public releases of large language models (LLMs) like ChatGPT and Claude (AI). These models exhibit human-like traits of knowledge, attention, and creativity, and have been integrated into various sectors, fueling exponential investment in AI. However, concerns about the potential risks and ethical implications of advanced AI have also emerged, causing debate about the future of AI and its impact on society.

Mechanical men and artificial beings appear in Greek myths, such as the golden robots of Hephaestus, Talos the bronze man, and Pygmalion's Galatea. In the Middle Ages, there were rumors of secret mystical or alchemical means of placing mind into matter, such as Jabir ibn Hayyan's Takwin, Paracelsus' homunculus, Rabbi Judah Loew's Golem and Roger Bacon's brazen head. By the 19th century, ideas about artificial men and thinking machines became a popular theme in fiction. Notable works include Mary Shelley's Frankenstein (1818), Johann Wolfgang von Goethe's, Faust, Part Two (1832), and Karel Čapek's R.U.R. (Rossum's Universal Robots) (1921). Stories of these creatures and their fates consider many of the same hopes, fears and ethical concerns that are presented by modern artificial intelligence. Issues relevant to AI were also discussed in speculative essays such as Samuel Butler's "Darwin among the Machines" (1863).

Realistic humanoid automata were built by craftsmen from many civilizations, including Yan Shi, Hero of Alexandria, Al-Jazari, Haroun al-Rashid, Jacques de Vaucanson, Leonardo Torres y Quevedo, Pierre Jaquet-Droz and Wolfgang von Kempelen. The oldest known automata were sacred statues of ancient Egypt and Greece. The faithful believed that craftsman had imbued these figures with very real minds, capable of wisdom and emotion—Hermes Trismegistus wrote that "by discovering the true nature of the gods, man has been able to reproduce it".

Artificial intelligence is based on the assumption that the process of human thought can be mechanized. Chinese, Indian and Greek philosophers developed structured methods of formal reasoning by the first millennium BCE. Formal logic was invented and improved by Greek, Islamic and European philosophers, such as Aristotle, Euclid, Al-Khwarizmi, Duns Scotus and René Descartes. Spanish philosopher Ramon Llull (1232–1315) developed several logical machines devoted to the production of knowledge by logical means; Llull described his machines as mechanical entities that could combine basic and undeniable truths by simple logical operations, produced by the machine by mechanical meanings, in such ways as to produce all the possible knowledge. Llull's work had a great influence on Gottfried Leibniz, who redeveloped his ideas.

2. Related Work, Literature Survey & Algorithmic Foundations

In the 17th century, Leibniz, Thomas Hobbes and René Descartes explored the possibility that all rational thought could be made as systematic as algebra or geometry. Hobbes wrote in Leviathan: "For reason ... is nothing but reckoning, that is adding and subtracting". Leibniz described a universal language of reasoning, the characteristica universalis, which would reduce argumentation to calculation so that "there would be no more need of disputation between two philosophers than between two accountants. For it would suffice to take their pencils in hand, down to their slates, and to say to each other (with a friend as witness, if they liked): Let us calculate." Boole's The Laws of Thought (1854) and Frege's Begriffsschrift (1879) defined the modern form of symbolic mathematical logic. Building on Frege's system, Russell and Whitehead presented a formal treatment of the foundations of arithmetic in the Principia Mathematica in 1913. Inspired by their success, David Hilbert challenged mathematicians of the 1920s and 30s to formalize all mathematical reasoning. In response, Gödel's incompleteness proof, Turing's machine and Church's Lambda calculus showed that there were, in fact, limits to what formal mathematics could do. However, within these limits, the Church-Turing thesis implied that a mechanical device, shuffling symbols as simple as 0 and 1, could imitate any conceivable process of mathematical reasoning or problem solving. The key insight was the Turing machine—a simple theoretical construct that captured the essence of abstract symbol manipulation. This device and the ideas behind it would inspire engineers and mathematicians of the 1940s to build machines that could theoretically carry out any form of formal reasoning or problem solving.

Calculating machines were designed or built in antiquity and throughout history by many people, including Gottfried Leibniz, Joseph Marie Jacquard, Charles Babbage, Percy Ludgate, Leonardo Torres Quevedo, Vannevar Bush, and others. Ada Lovelace speculated that Babbage's machine was "a thinking or ... reasoning machine", but warned "It is desirable to guard against the possibility of exaggerated ideas that arise as to the powers" of the machine. The first modern computers were the massive machines of the Second World War (such as Konrad Zuse's Z3, Tommy Flowers' Heath Robinson and Colossus, Atanasoff and Berry's ABC, and ENIAC at the University of Pennsylvania). ENIAC was based on the theoretical foundation laid by Alan Turing and developed by John von Neumann, and proved to be the most influential.

The earliest research into thinking machines was inspired by a network of ideas that became prevalent in the late 1930s, 1940s, and early 1950s. Recent research in neurology had shown that the brain was an electrical network of neurons that fired in all-or-nothing pulses. Norbert Wiener's cybernetics described control and stability in electrical networks. Claude Shannon's information theory described digital signals (i.e., all-or-nothing signals). Alan Turing's theory of computation showed that any form of computation could be described digitally. The close relationship between these ideas suggested that it might be possible to construct an "electronic brain". In the 1940s and 50s, a handful of scientists from a variety of fields (mathematics, psychology, engineering, economics and political science) explored several research directions that would be vital to later AI research. Alan Turing was among the first people to seriously investigate the theoretical possibility of "machine intelligence". The field of "artificial intelligence research" was founded as an academic discipline in 1956.

In 1950, Turing published a landmark paper, "Computing Machinery and Intelligence", in which he speculated about the possibility of creating machines that think. In the paper, he noted that "thinking" is difficult to define and devised his famous Turing test: If a machine could carry on a conversation (over a teleprinter) that was indistinguishable from a conversation with a human being, then it was reasonable to say that the machine was "thinking". This simplified version of the problem allowed Turing to argue convincingly that a "thinking machine" was at least plausible and the paper answered all the most common objections to the proposition. The Turing test was the first serious proposal in the philosophy of artificial intelligence.

3. Dataset Ingestion, Preprocessing & Stochastic Augmentations

When access to digital computers became possible in the mid-fifties, a few scientists instinctively recognized that a machine that could manipulate numbers could also manipulate symbols and that the manipulation of symbols could well be the essence of human thought. This was a new approach to creating thinking machines. In 1955, Allen Newell and future Nobel Laureate Herbert A. Simon created the "Logic Theorist", with help from J. C. Shaw. The program would eventually prove 38 of the first 52 theorems in Russell and Whitehead's Principia Mathematica, and find new and more elegant proofs for some. Simon said that they had "solved the venerable mind/body problem, explaining how a system composed of matter can have the properties of mind." The symbolic reasoning paradigm they introduced would dominate AI research and funding until the mid-90s, as well as inspire the cognitive revolution.

The Dartmouth workshop of 1956 was a pivotal event that marked the formal inception of AI as an academic discipline. It was organized by Marvin Minsky and John McCarthy, with the support of two senior scientists Claude Shannon and Nathan Rochester of IBM. The proposal for the conference stated they intended to test the assertion that "every aspect of learning or any other feature of intelligence can be so precisely described that a machine can be made to simulate it". The term "Artificial Intelligence" was introduced by John McCarthy at the workshop. The participants included Ray Solomonoff, Oliver Selfridge, Trenchard More, Arthur Samuel, Allen Newell and Herbert A. Simon, all of whom would create important programs during the first decades of AI research. At the workshop, Newell and Simon debuted the "Logic Theorist". The workshop was the moment that AI gained its name, its mission, its first major success and its key players, and is widely considered the birth of AI.

In the autumn of 1956, Newell and Simon also presented the Logic Theorist at a meeting of the Special Interest Group in Information Theory at the Massachusetts Institute of Technology (MIT). At the same meeting, Noam Chomsky discussed his generative grammar, and George Miller described his landmark paper "The Magical Number Seven, Plus or Minus Two". Miller wrote "I left the symposium with a conviction, more intuitive than rational, that experimental psychology, theoretical linguistics, and the computer simulation of cognitive processes were all pieces from a larger whole." This meeting was the beginning of the "cognitive revolution"—an interdisciplinary paradigm shift in psychology, philosophy, computer science and neuroscience. It inspired the creation of the sub-fields of symbolic artificial intelligence, generative linguistics, cognitive science, cognitive psychology, cognitive neuroscience and the philosophical schools of computationalism and functionalism. All these fields used related tools to model the mind and results discovered in one field were relevant to the others. The cognitive approach allowed researchers to consider "mental objects" like thoughts, plans, goals, facts or memories, often analyzed using high level symbols in functional networks. These objects had been forbidden as "unobservable" by earlier paradigms such as behaviorism and positivism. Symbolic mental objects would become the major focus of AI research and funding for the next several decades.

An important goal of AI research is to allow computers to communicate in natural languages like English. An early success was Daniel Bobrow's program STUDENT, which could solve high-school algebra word-problems. A semantic net represents concepts (e.g., "house", "door") as nodes, and relations among concepts as links between the nodes (e.g. "has-a"). Ross Quillian wrote the first AI program to use a semantic net, and the most successful (and controversial) version was Roger Schank's Conceptual dependency theory. Joseph Weizenbaum's ELIZA could carry out conversations that were so realistic that users occasionally were fooled into thinking they were communicating with a human being and not with a computer program (see ELIZA effect). But in fact, ELIZA simply gave a canned response or repeated back what was said to it, rephrasing its response with a few grammar-rules. ELIZA was the first chatbot.

4. Neural Network Backbone, Layer Specifications & Loss Formulations

In the 1960s, research-funders primarily supported laboratories researching symbolic AI, however, a few laboratories pursued research in neural networks.

The perceptron, a single-layer neural network was introduced in 1958 by Frank Rosenblatt (who had been a schoolmate of Marvin Minsky at the Bronx High School of Science). Like most AI researchers, he was optimistic about their power, predicting that a perceptron "may eventually be able to learn, make decisions, and translate languages". Rosenblatt was primarily funded by the Office of Naval Research. Bernard Widrow and his student Ted Hoff built ADALINE (1960) and MADALINE (1962), which had up to 1000 adjustable weights. A group at Stanford Research Institute led by Charles A. Rosen and Alfred E. (Ted) Brain built two neural-network machines named MINOS I (1960) and II (1963), mainly funded by the U.S. Army Signal Corps. MINOS II had 6600 adjustable weights, and was controlled with an SDS 910 computer in a configuration named MINOS III (1968), which could classify symbols on army maps, and recognize hand-printed characters on Fortran coding sheets. Most of neural-network research during this early period involved building and using bespoke hardware, rather than simulation on digital computers. However, partly due to lack of results and partly due to competition from symbolic AI research, the MINOS project ran out of funding in 1966. Rosenblatt failed to secure continued funding in the 1960s. In 1969, research came to a sudden halt with the publication of Minsky and Papert's 1969 book Perceptrons. It suggested that there were severe limitations to what perceptrons could do and that Rosenblatt's predictions had been grossly exaggerated. The effect of the book was that virtually no research was funded in connectionism for 10 years. The competition for government funding ended with the victory of symbolic AI approaches over neural networks. Minsky (who had worked on SNARC) became a staunch objector to pure connectionist AI. Widrow (who had worked on ADALINE) turned to adaptive signal processing. The SRI group (which worked on MINOS) turned to symbolic AI and robotics. The main problem was the inability to train multilayered networks (versions of backpropagation had already been used in other fields, but it was unknown to these researchers). The AI community became aware of backpropagation in the 1980s, and, in the 21st century, neural networks would become enormously successful, fulfilling all of Rosenblatt's optimistic predictions. Rosenblatt did not live to see this, however, as he died in a boating accident in 1971.

1958, H. A. Simon and Allen Newell: "within ten years a digital computer will be the world's chess champion" and "within ten years a digital computer will discover and prove an important new mathematical theorem." 1965, H. A. Simon: "machines will be capable, within twenty years, of doing any work a man can do." 1967, Marvin Minsky: "Within a generation... the problem of creating 'artificial intelligence' will substantially be solved." 1970, Marvin Minsky (in Life magazine): "In from three to eight years we will have a machine with the general intelligence of an average human being."

Limited computer power: There was not enough memory or processing speed to accomplish anything truly useful. For example: Ross Quillian's successful work on natural language was demonstrated with a vocabulary of only 20 words, because that was all that would fit in memory. Hans Moravec argued in 1976 that computers were still millions of times too weak to exhibit intelligence. He suggested an analogy: artificial intelligence requires computer power in the same way that aircraft require horsepower. Below a certain threshold, it's impossible, but, as power increases, eventually it could become easy. "With enough horsepower," he wrote, "anything will fly". Intractability and the combinatorial explosion: In 1972 Richard Karp (building on Stephen Cook's 1971 theorem) showed there are many problems that can only be solved in exponential time. Finding optimal solutions to these problems requires extraordinary amounts of computer time, except when the problems are trivial. This limitation applied to all symbolic AI programs that used search trees and meant that many of the "toy" solutions used by AI would never scale to useful systems. Moravec's paradox: Early AI research had been very successful at getting computers to do "intelligent" tasks like proving theorems, solving geometry problems and playing chess. Their success at these intelligent tasks convinced them that the problem of intelligent behavior had been largely solved. However, they utterly failed to make progress on "unintelligent" tasks like recognizing a face or crossing a room without bumping into anything. By the 1980s, researchers would realize that symbolic reasoning was utterly unsuited for these perceptual and sensorimotor tasks and that there were limits to this approach. The breadth of commonsense knowledge: Many important artificial intelligence applications like vision or natural language require enormous amounts of information about the world: the program needs to have some idea of what it might be looking at or what it is talking about. This requires that the program know most of the same things about the world that a child does. Researchers soon discovered that this was a vast amount of information with billions of atomic facts. No one in 1970 could build a database large enough and no one knew how a program might learn so much information. Representing commonsense reasoning: Several related problems appeared when researchers tried to represent commonsense reasoning using formal logic or symbols. Descriptions of very ordinary deductions tended to get longer and longer the more one worked on them, as more and more exceptions, clarifications and distinctions were required. However, when people thought about ordinary concepts, they did not rely on precise definitions, rather they seemed to make hundreds of imprecise assumptions, correcting them when necessary using their entire body of commonsense knowledge. Gerald Sussman observed that "using precise language to describe essentially imprecise concepts doesn't make them any more precise."

5. Training Protocol, Convergence Behavior & Experimental Evaluation

The agencies that funded AI research, such as the British government, DARPA and the National Research Council (NRC) became frustrated with the lack of progress and eventually cut off almost all funding for undirected AI research. The pattern began in 1966 when the Automatic Language Processing Advisory Committee (ALPAC) report criticized machine translation efforts. After spending $20 million, the NRC ended all support. In 1973, the Lighthill report on the state of AI research in the UK criticized the failure of AI to achieve its "grandiose objectives" and led to the dismantling of AI research in that country. (The report specifically mentioned the combinatorial explosion problem as a reason for AI's failings.) DARPA was deeply disappointed with researchers working on the Speech Understanding Research program at CMU and canceled an annual grant of $3 million. Hans Moravec blamed the crisis on the unrealistic predictions of his colleagues. "Many researchers were caught up in a web of increasing exaggeration." However, there was another issue: since the passage of the Mansfield Amendment in 1969, DARPA had been under increasing pressure to fund "mission-oriented direct research, rather than basic undirected research". Funding for the creative, freewheeling exploration that had gone on in the 60s would not come from DARPA, which instead directed money at specific projects with clear objectives, such as autonomous tanks and battle management systems. The major laboratories (MIT, Stanford, CMU and Edinburgh) had been receiving generous support from their governments, and when it was withdrawn, these were the only places that were seriously impacted by the budget cuts. The thousands of researchers outside these institutions and the many thousands that were joining the field were unaffected.

Several philosophers had strong objections to the claims being made by AI researchers. One of the earliest was John Lucas, who argued that Gödel's incompleteness theorem showed that a formal system (such as a computer program) could never see the truth of certain statements, while a human being could. Hubert Dreyfus ridiculed the broken promises of the 1960s and critiqued the assumptions of AI, arguing that human reasoning actually involved very little "symbol processing" and a great deal of embodied, instinctive, unconscious "know how". John Searle's Chinese Room argument, presented in 1980, attempted to show that a program could not be said to "understand" the symbols that it uses (a quality called "intentionality"). If the symbols have no meaning for the machine, Searle argued, then the machine can not be described as "thinking". These critiques were not taken seriously by AI researchers. Problems like intractability and commonsense knowledge seemed much more immediate and serious. It was unclear what difference "know how" or "intentionality" made to an actual computer program. MIT's Minsky said of Dreyfus and Searle, "they misunderstand, and should be ignored." Dreyfus, who also taught at MIT, was given a cold shoulder: he later said that AI researchers "dared not be seen having lunch with me." Joseph Weizenbaum, the author of ELIZA, was also an outspoken critic of Dreyfus' positions, but he "deliberately made it plain that [his AI colleagues' treatment of Dreyfus] was not the way to treat a human being," and was unprofessional and childish. Weizenbaum began to have serious ethical doubts about AI when Kenneth Colby wrote a "computer program which can conduct psychotherapeutic dialogue" based on ELIZA. Weizenbaum was disturbed that Colby saw a mindless program as a serious therapeutic tool. A feud began, and the situation was not helped when Colby did not credit Weizenbaum for his contribution to the program. In 1976, Weizenbaum published Computer Power and Human Reason which argued that the misuse of artificial intelligence has the potential to devalue human life.

In 1982, physicist John Hopfield was able to prove that a form of neural network (now called a "Hopfield net") could learn and process information, and provably converges after enough time under any fixed condition. It was a breakthrough, as it was previously thought that nonlinear networks would, in general, evolve chaotically. Geoffrey Hinton proved a similar result about a device called a "Boltzmann machine". (Hopfield and Hinton would eventually receive the 2024 Nobel prize for this work.) In 1986, Hinton and David Rumelhart popularized a method for training neural networks called "backpropagation". These three developments helped to revive the exploration of artificial neural networks. Neural networks, along with several other similar models, received widespread attention after the 1986 publication of the Parallel Distributed Processing, a two-volume collection of papers edited by Rumelhart and psychologist James McClelland. The new field was named "connectionism" which was contrasted with symbolic AI. Hinton called symbols the "luminous aether of AI"―that is, an unworkable and misleading model of intelligence. In 1990, Yann LeCun at Bell Labs used convolutional neural networks to recognize handwritten digits. The system was used widely in the 90s, reading zip codes and personal checks. This was the first genuinely useful application of neural networks.

Rodney Brooks, Hans Moravec and others argued that, to show real intelligence, a machine needs to have a body—it needs to perceive, move, survive, and deal with the world. Sensorimotor skills are essential to higher level skills such as commonsense reasoning. They can't be efficiently implemented using abstract symbolic reasoning, so AI should solve the problems of perception, mobility, manipulation and survival without using symbolic representation at all. These robotics researchers advocated building intelligence "from the bottom up". A precursor to this idea was David Marr, who had come to MIT in the late 1970s from a successful background in theoretical neuroscience to lead the group studying vision. He rejected all symbolic approaches (both McCarthy's logic and Minsky's frames), arguing that AI needed to understand the physical machinery of vision from the bottom up before any symbolic processing took place. (Marr's work would be cut short by leukemia in 1980.) In his 1990 paper "Elephants Don't Play Chess", robotics researcher Brooks took direct aim at the physical symbol system hypothesis, arguing that symbols are not always necessary since "the world is its own best model. It is always exactly up to date. It always has every detail there is to be known. The trick is to sense it appropriately and often enough." In the 1980s and 1990s, many cognitive scientists also rejected the symbol processing model of the mind and argued that the body was essential for reasoning, a theory called the "embodied mind thesis".