Some Predictions about Generative AI and LLMs

Photo by Christopher Burns on Unsplash

This is a work in progress….

Recent advancements in Generative AI, particularly in the realm of Large Language Models (LLMs), have showcased remarkable performance across various domains, potentially heralding the onset of a transformative revolution on par with the industrial revolution of the 18th century. Nonetheless, it would be premature to declare outright victory, given the existence of numerous risks and limitations. Following the previous post, this article aims to articulate a series of contemplations on this matter. I share some of my thoughts in this article.

1. AI encompasses a wide range of subfields beyond Generative AI, and it is important to recognize that it is not solely defined by Large Language Models (LLMs). AI is poised for continued growth and advancement as a discipline, while conventional Machine Learning (ML) will persist as a valuable component. In the realm of AI, LLMs are like an investment, but conventional ML serves as an insurance policy, providing transparency, explainability, requiring less training data, and incurring lower compute costs.
2. Although there is a potential risk of a third AI winter being triggered by the proliferation of Large Language Models (LLMs), the likelihood of such an occurrence is currently diminishing. It is crucial to remain cognizant of the fact that the previous instances of AI winters were predominantly caused by excessive investments and exaggerated promises rather than being impeded by technical or scientific challenges. It is worth noting that a similar pattern can also be observed in recent advancements in AI.
3. The growth trajectory of Large Language Models (LLMs) is expected to persist. However, it is worth noting that many smaller LLMs will likely fade away after the initial excitement subsides and the hype diminishes. A few dominant players will ultimately emerge and capture the majority of the market share, conforming to a power law distribution. Nevertheless, there will also be a long tail comprising numerous small companies specializing in providing niche LLM solutions, such as vertical LLMs tailored for specific industries or domains.
4. The democratization of access to Large Language Models (LLMs) will occur, following a similar pattern to the democratization of the web, social networks, and cloud computing. However, it is important to differentiate between the democratization of access and the democratization of LLM adaptations and market share. In the case of LLMs, democratization will primarily apply to the accessibility of the technology itself rather than the broader aspects of adaptation and market dominance. It is crucial to understand that democratization operates on four distinct layers: technology, adaptation, business models, and applications. Among these layers, only the application layer can truly be democratized, analogous to the widespread usage of cell phones compared to the control exerted by specific cell phone suppliers. Additionally, it should be noted that the concept of a “distributed service” does not inherently imply democratization.
5. The disparity between the industrialized and developing nations is likely to widen further, giving rise to what can be termed the “AI Gap.” Many developing countries face significant challenges due to inadequate infrastructure for AI development and implementation. This lack of infrastructure creates a substantial barrier to their participation and progress in the field of AI.
6. Governments are expected to eventually initiate regulation of the AI industry, albeit potentially at a later stage. Several aspects within the AI domain can be subject to regulation, including:

• Data attribution: Regulations may govern the use of public data to ensure that building a product based on such data does not lead to unauthorized monetization.
• Privacy protection policies: Regulations can address protecting individuals’ privacy rights in the context of AI applications and data handling.
• Foreign government influence: Measures may be implemented to mitigate the influence exerted by foreign governments on AI systems and ensure the integrity and independence of AI technologies.
• Deepfake and truth manipulation: Regulations could be introduced to address the challenges posed by deep fake technology and the manipulation of information, aiming to preserve the integrity of truth and combat misinformation.
• Plagiarism and authorship: Regulatory frameworks may be established to address plagiarism issues and establish clear guidelines on authorship and intellectual property rights within the AI field.
• Limiting parameters: Regulations might limit the number of parameters used in AI models, aiming to mitigate potential risks associated with excessively complex and resource-intensive models.
• Greener AI policies: Regulations could promote environmentally friendly practices in developing and deploying AI systems, ensuring energy efficiency and reducing the carbon footprint associated with AI technologies.
• Service democratization: Measures can be taken to foster the democratization of AI services, ensuring equal access and opportunities for individuals and businesses to benefit from AI technologies.

7. The widespread adoption of generative AI holds immense potential, offering new opportunities in various areas:

• Personal AI: The extensive use of generative AI will bring “Personal AI” into our daily lives, having a transformative impact similar to the advent of personal computers (PCs) throughout history.
• Language barriers will become history: Generative AI will play a pivotal role in breaking down language barriers, making communication across different languages more seamless and accessible.
• Education revolution: The field of education will experience a revolution in content development, delivery, and assessment through the integration of generative AI. This innovation will enhance learning experiences and open up new possibilities for personalized education.
• Elevating creativity: Similar to how the invention of calculators did not diminish the importance of mathematics but rather elevated it to new levels, Large Language Models (LLMs) will not stifle creativity but enable individuals to explore creativity at higher levels of abstraction.
• Singularity myth: It is important to recognize that generative AI and LLMs are not inherently dangerous. The risks arise from how people choose to utilize and control these technologies. As long as AI lacks moral agency, it does not pose an existential threat to humanity.
• Constant evolution: LLMs are dynamic and continuously evolving due to their relatively new nature. This ongoing evolution presents opportunities for further advancements and improvements.
• Model imitation: Despite existing legal hurdles, pursuing model imitation remains a valuable endeavour. Leveraging proprietary LLMs to fine-tune or pre-train smaller-scale open-source models or conventional machine learning approaches is a viable approach that offers practical benefits.
• Scientific research: LLMs have the capacity to streamline the laborious and tiresome processes inherent in scientific research, such as conducting comprehensive literature surveys. Nonetheless, a potential drawback of their extensive adoption by scientists in scientific reports is an increased tendency towards conformity, which may come at the cost of genuine authenticity.

8. Large Language Models (LLMs) present certain risks and challenges that must be addressed. Some of the risks associated with LLMs include:

• Hulecination occurs when the model produces responses or information that may sound plausible but are not accurate or supported by reliable sources. Hallucination can be unintentional and arise due to the model's training data or limitations in understanding context, or it can result from the model trying to generate a response without having access to real-time information or the ability to verify facts.
• Fake news, miss information fabrication, political influence on the scale: LLMs are designed to generate human-like text based on the patterns and information they have learned from their training data. While efforts are made to ensure the accuracy and reliability of the responses, there is still a possibility of generating incorrect or misleading information.
• Bias in all demographic and socio-political spectrums: Bias can arise from various sources, including the training data and the algorithms used during development. Bias refers to systematic favouritism or prejudice towards particular groups or perspectives, which can manifest in the generated responses.
• Shortcut learning refers to a phenomenon where the model relies on simple patterns or shortcuts in the training data instead of deeply understanding the underlying concepts. It can result in the model providing plausible-sounding but incorrect or misleading responses.
• Lack of reasoning: LLMs may lack the ability to reason deductively, inductively, or abductively, limiting their capacity for logical and critical thinking.
• Ethical and moral issues: Using LLMs raises ethical concerns, such as the responsible use of AI, potential misuse, and unintended consequences.
• Socio-economic impact: The widespread adoption of LLMs may lead to job losses in certain industries while simultaneously creating new jobs that require expertise in emerging AI technologies, leading to socio-economic disruptions.
• Non-democratic access: Access to advanced LLM models may be limited to larger institutions, creating an imbalance against smaller schools and research labs.
• Data dependency: LLMs heavily rely on data, and the availability and quality of data can significantly impact their performance. Curated domain-specific data may be necessary to enhance the performance of LLMs.
• LLM maximalism: LLM is a generative AI technology, and its first objective is to generate coherent, contextually appropriate text, not factual accuracy. For the majority of problems in machine learning, there is a small amount of training data and limited computing resources. For these cases, conventional supervised learning should be tested first.
• Large Language Models (LLMs) are valuable tools for rapid prototyping and Proof of Concepts (POCs). However, deploying LLMs in production environments can be challenging. This is primarily because LLMs extend beyond the scope of non-generative tasks, such as classification problems. In such cases, supervised learning approaches tend to yield superior results compared to in-context learning and prompt engineering associated with LLMs. Therefore, while LLMs have their merits, alternative approaches may be more effective for achieving optimal outcomes in production settings, particularly for non-generative tasks like classification.
• Prompt engineering indeed presents an appealing prospect as it enables a user-friendly experience by interacting with models in natural language. However, there are two significant challenges associated with prompt engineering stemming from the nature of Large Language Models (LLMs). The first challenge is that LLMs can produce ambiguous results. Due to the vast amount of data, they are trained on, LLMs may generate responses with multiple interpretations or lack clarity in certain contexts. This ambiguity can lead to confusion or incorrect interpretations of the model’s output. The second challenge lies in the potential inconsistency of LLM outputs, even when provided with identical prompts. This inconsistency arises because LLMs are stochastic models incorporating random elements into their decision-making process. Consequently, the same prompt may yield different outputs from the LLM on different occasions, making it challenging to guarantee consistent results. These two critical problems, the ambiguity of LLM outputs and their inherent inconsistency, pose limitations to the reliability and predictability of LLMs in the context of prompt engineering. While using LLMs can provide a natural language interface, it is essential to consider and address these challenges to ensure accurate and consistent user experiences.

For further reading, see my reading list.