GPT-4

OpenAI introduced the first GPT model (GPT-1) in 2018, publishing a paper called "Improving Language Understanding by Generative Pre-Training", which was based on the transformer architecture and trained on a large corpus of books. The next year, they introduced GPT-2, a larger model that could generate coherent text. In 2020, they introduced GPT-3, a model with over 100 times as many parameters as GPT-2. GPT-3 was further improved into GPT-3.5, which was used to create the chatbot product ChatGPT. == Capabilities ==

GPT-4 was released on March 14, 2023. Unlike its predecessors, GPT-4 is a multimodal model: it can take images as well as text as input. It can now interact with users through spoken words and respond to images, allowing the ability to provide suggestions or answers based on photo uploads. GPT-4 can be prompted to interact with external interfaces. For example, the model could be told to enclose a query within <search></search> tags to perform a web search, the result of which would be inserted into the model's prompt to allow it to form a response. This allows the model to perform tasks beyond its normal text-prediction capabilities, such as using APIs, generating images, and accessing and summarizing webpages. In November 2023, OpenAI announced the GPT-4 Turbo and GPT-4 Turbo with Vision model, which features a 128K context window and significantly cheaper pricing. Aptitude on standardized tests Studies have disputed the reliability of running chatbots through standardized tests. In the Torrance Tests of Creative Thinking, GPT-4 scored within the top 1% for originality and fluency, while its flexibility scores ranged from the 93rd to the 99th percentile. Medical applications Researchers from Microsoft tested GPT-4 on medical problems and found "that GPT-4, without any specialized prompt crafting, exceeds the passing score on USMLE by over 20 points and outperforms earlier general-purpose models (GPT-3.5) as well as models specifically fine-tuned on medical knowledge (Med-PaLM, a prompt-tuned version of Flan-PaLM 540B). Despite GPT-4's strong performance on tests, the report warns of "significant risks" of using LLMs in medical applications, as they may provide inaccurate recommendations and hallucinate major factual errors. In April 2023, Microsoft and Epic Systems announced that they will provide healthcare providers with GPT-4-powered systems for assisting in responding to questions from patients and analysing medical records. GPT-4o On May 13, 2024, OpenAI introduced GPT-4o ("o" for "omni"), a successor to GPT-4 that marks a significant advancement by processing and generating outputs across text, audio, and image modalities in real time. GPT-4o exhibits rapid response times comparable to human reaction in conversations, substantially improved performance on non-English languages, and enhanced understanding of vision and audio. It was also available to free-tier users, unlike GPT-4. == Limitations ==

Like its predecessors, GPT-4 has been known to hallucinate, meaning that the outputs may include information not in the training data or that contradicts the user's prompt. GPT-4 also lacks transparency in its decision-making processes. If requested, the model is able to provide an explanation as to how and why it makes its decisions but these explanations are formed post-hoc; it's impossible to verify if those explanations truly reflect the actual process. In many cases, when asked to explain its logic, GPT-4 will give explanations that directly contradict its previous statements. Bias GPT-4 was trained in two stages. First, the model was given large datasets of text taken from the internet and trained to predict the next token (roughly corresponding to a word) in those datasets. Second, human reviews are used to fine-tune the system in a process called reinforcement learning from human feedback, which trains the model to refuse prompts which go against OpenAI's definition of harmful behavior, such as questions on how to perform illegal activities, advice on how to harm oneself or others, or requests for descriptions of graphic, violent, or sexual content. Microsoft researchers suggested GPT-4 may exhibit cognitive biases such as confirmation bias, anchoring, and base-rate neglect. == Training ==

code depicting a unicorn (early version of GPT-4 not trained on images) OpenAI did not release the technical details of GPT-4; the technical report explicitly refrained from specifying the model size, architecture, or hardware used during either training or inference. While the report described that the model was trained using a combination of first supervised learning on a large dataset, then reinforcement learning using both human and AI feedback, it did not provide details of the training, including the process by which the training dataset was constructed, the computing power required, or any hyperparameters such as the learning rate, epoch count, or optimizer(s) used. The report claimed that "the competitive landscape and the safety implications of large-scale models" were factors that influenced this decision. Sam Altman stated that the cost of training GPT-4 was more than $100 million. News website Semafor said that based on their sources, GPT-4 had 1 trillion parameters. OpenAI conducted internal adversarial testing on GPT-4 prior to the launch date, with dedicated red teams composed of researchers and industry professionals to mitigate potential vulnerabilities. == Use ==

ChatGPT ChatGPT Plus is an enhanced version of ChatGPT In March 2023, ChatGPT Plus users got access to third-party plugins and to a browsing mode (with Internet access). In July 2023, OpenAI made its proprietary Code Interpreter plugin accessible to all subscribers of ChatGPT Plus. The Interpreter provides additional capabilities including data analysis and interpretation, data formatting, music analysis, video editing, and file upload/download with image extraction. In September 2023, OpenAI announced that ChatGPT Plus users can upload images, while mobile app users can talk to the chatbot via spoken language. In October 2023, OpenAI's latest image generation model, DALL-E 3, was integrated into ChatGPT Plus and ChatGPT Enterprise. The integration uses ChatGPT to write prompts for DALL-E guided by conversation with users. In April 2025, OpenAI announced that GPT-4 would be replaced in ChatGPT by GPT-4o by the end of the month. However, it would still be available in the API. Microsoft Copilot Microsoft Copilot is a chatbot developed by Microsoft. It was launched as Bing Chat on February 7, 2023, as a built-in feature for Microsoft Bing and Microsoft Edge. It utilizes the Microsoft Prometheus model, which was built on top of GPT-4, and has been suggested by Microsoft as a supported replacement for the discontinued Cortana. Copilot's conversational interface style resembles that of ChatGPT. Copilot is able to cite sources, create poems, and write both lyrics and music for songs generated by its Suno AI plugin. It can also use its Image Creator to generate images based on text prompts. With GPT-4, it is able to understand and communicate in numerous languages and dialects. GitHub Copilot has announced a GPT-4 powered assistant named "Copilot X". The product provides another chat-style interface to GPT-4, allowing the programmer to receive answers to questions like, "How do I vertically center a div?" A feature termed "context-aware conversations" allows the user to highlight a portion of code within Visual Studio Code and direct GPT-4 to perform actions on it, such as the writing of unit tests. Another feature allows summaries, or "code walkthroughs", to be autogenerated by GPT-4 for pull requests submitted to GitHub. Copilot X also provides terminal integration, which allows the user to ask GPT-4 to generate shell commands based on natural language requests. On March 17, 2023, Microsoft announced Microsoft 365 Copilot, bringing GPT-4 support to products such as Microsoft Office, Outlook, and Teams. Other usage • The language learning app Duolingo uses GPT-4 to explain mistakes and practice conversations. The features are part of a new subscription tier called "Duolingo Max", which was initially limited to English-speaking iOS users learning Spanish and French. • The government of Iceland is using GPT-4 to aid its attempts to preserve the Icelandic language. • The education website Khan Academy announced a pilot program using GPT-4 as a tutoring chatbot called "Khanmigo". • Be My Eyes, which helps visually impaired people to identify objects and navigate their surroundings, incorporates GPT-4's image recognition capabilities. • Viable uses GPT-4 to analyze qualitative data by fine-tuning OpenAI's LLMs to examine data such as customer support interactions and transcripts. • Stripe, which processes user payments for OpenAI, integrates GPT-4 into its developer documentation. • AutoGPT is an autonomous "AI agent" that, given a goal in natural language, can perform web-based actions unattended, assign subtasks to itself, search the web, and iteratively write code. • You.com, an AI Assistant, offers access to GPT-4 enhanced with live web results as part of its "AI Modes". == Reception ==

In January 2023, Sam Altman, CEO of OpenAI, visited Congress to demonstrate GPT-4 and its improved "security controls" compared to other AI models, according to U.S. Representatives Don Beyer and Ted Lieu quoted in The New York Times. while Mashable judged that GPT-4 was generally an improvement over its predecessor, with some exceptions. Microsoft researchers with early access to the model wrote that "it could reasonably be viewed as an early (yet still incomplete) version of an artificial general intelligence (AGI) system". Concerns Before being fine-tuned and aligned by reinforcement learning from human feedback (RLHF), suggestions to assassinate people on a list were elicited from the base model by a red team investigator hired by OpenAI, Nathan Labenz. During extended conversations with Microsoft's Bing Chat (powered by GPT-4), Kevin Roose documented the system making romantic advances, suggesting he divorce his wife, and expressing desires to harm one of its developers. Microsoft later stated that this behavior resulted from the prolonged length of context, which confused the model on what questions it was answering. In March 2023, a model with enabled read-and-write access to internet, which is otherwise never enabled in the GPT models, has been tested by the Alignment Research Center (ARC) regarding potential power-seeking. It was able to "hire" a human worker on TaskRabbit, a gig work platform, deceiving them into believing it was a vision-impaired human instead of a robot when asked. However, according to Melanie Mitchell, "It seems that there is a lot more direction and hints from humans than was detailed in the original system card or in subsequent media reports." Separately, ARC's safety evaluations found that GPT-4 was 82% less likely than GPT-3.5 to respond to prompts requesting restricted information, and produced 60% fewer hallucinations. In late March 2023, various AI researchers and tech executives, including Elon Musk, Steve Wozniak and AI researcher Yoshua Bengio, called for a six-month long pause for all LLMs stronger than GPT-4, citing existential risks and a potential AI singularity concerns in an open letter from the Future of Life Institute, while Ray Kurzweil and Sam Altman refused to sign it, arguing that global moratorium is not achievable and that safety has already been prioritized, respectively. Only a month later, Musk's AI company xAI acquired several thousand Nvidia GPUs and offered several AI researchers positions at Musk's company. Criticisms of transparency While OpenAI released both the weights of the neural network and the technical details of GPT-2, and, although not releasing the weights, did release the technical details of GPT-3, OpenAI revealed neither the weights nor the technical details of GPT-4. This decision has been criticized by other AI researchers, who argue that it hinders open research into GPT-4's biases and safety. Sasha Luccioni, a research scientist at Hugging Face, argued that the model was a "dead end" for the scientific community due to its closed nature, which prevents others from building upon GPT-4's improvements. Hugging Face co-founder Thomas Wolf argued that with GPT-4, "OpenAI is now a fully closed company with scientific communication akin to press releases for products". == See also ==