Generative AI Gold Rush

🗞️ The Tech Issue | January 10, 2024

Author

Qamar Zia
January 10, 2024

☕️ Greetings, and welcome to my daily dive into the Generative AI landscape.

Check out my profile to see where I'm coming from, and swing by INVENEW to see what I'm up to these days. Also, I've got this new blog, ReROAR, where I write about AI, the future of work and living, cool solopreneurship ideas, and the must-have skills in our AI-driven future. 

In today’s issue:

  • Can Language Models Replace Compilers?

  • Out-of-Office Emails Are Boring: Making Them Pop with Generative AI

  • OpenAI GPTs: 9 Problems That Make Them NO-GO For Businesses

  • Big Tech and Generative AI Gold Rush

  • How to train coding LLMs with small auto-generated datasets

  • Memory, Consciousness, and Large Language Model.

  • And more

🔔 Please forward this newsletter to your friends and team members and invite them to join. This will help me grow my reach. Thanks, Qamar.

Big Tech and Generative AI Gold Rush

Recent years have witnessed a surge in hype around various tech trends, with Generative AI currently at the forefront, drawing significant investments from Venture Capital and big tech firms. Despite historical trends of such technologies taking longer to mature than expected, Generative AI has seen rapid growth and high levels of funding in 2023. Big Tech companies, notably Microsoft, Google, and Nvidia, have aggressively invested in this space, raising questions about potential monopolization and the future landscape of the AI industry.

Key Points:

  • Generative AI Investment Rush: 2023 saw a significant influx of funding into Generative AI startups by both VC and BigTech companies.

  • Big Tech's Role: Microsoft, Google, Nvidia, and likely Amazon and Apple, are heavily investing in Generative AI, influencing the direction and control of this technology.

  • Nvidia's Diversified Investment: Nvidia shows a broad scope in funding AI companies, aligning with their CEO's prediction of AI competitiveness with humans within five years.

  • Large Language Model (LLM) Startups' Growth: Companies like OpenAI, Anthropic, and Cohere have seen substantial growth, with funding rounds increasing in size due to the computational demands.

  • Leadership and Control Issues: Controversies around leadership in AI companies reflect the complex dynamics of control and influence by VC, billionaires, and BigTech.

  • Generational Adoption and Trust: Gen Z shows high adoption and trust in Generative AI technology, indicating a shift in technological reliance and decision-making.

  • Potential Monopolization by Big Tech: There is an ongoing debate about whether Big Tech companies are monopolizing the AI boom, shaping the future of AI development and accessibility.

  • Geographical Differences in AI Adoption: Different adoption rates in countries like India suggest a diverse global impact and potential for Generative AI technology.

  • Future Prospects: The trajectory of Generative AI and its potential to revolutionize various sectors, from cloud computing to everyday tasks, remains a crucial area of observation and analysis.

Image Source: Charlie Guo

📰 Latest From The Web

Nice to Meet You! Speeding up Developer Onboarding with LLMs and Unblocked: Onboarding new developers is costly and time-consuming, often distracting senior team members. Unblocked, a Large Language Model (LLM), eases this by integrating with team-specific data sources like code and documentation. It provides quick, personalized assistance, streamlining the onboarding process and reducing the workload on senior developers. Read more at Hackernoon.com.

Can Language Models Replace Compilers?: Kevlin Henney discussed the potential of AI tools like GitHub Copilot to replace high-level languages with direct machine code generation. However, they concluded that AI-generated code currently lacks the necessary determinism and repeatability for complex software development. Read more at Oreilly.com.

Out-of-Office Emails Are Boring: Making Them Pop with Generative AI: The article details the development of a Python web-app that uses GPT-4 and DALL-E 3 to create unique out-of-office emails with accompanying images. The app, designed to add creativity to standard auto-replies, allows users to manage messages and set Outlook out-of-office settings automatically. It emphasizes user-friendly interface and automation, and the source code is available on GitHub. This innovative approach to mundane auto-replies demonstrates practical applications of Generative AI in workplace communication. Read more at Towardsdatascience.com.

OpenAI says it's "impossible" to train state-of-the-art models without copyrighted data: The lawsuit between The New York Times and OpenAI underscores the challenge of using copyrighted content for AI training. OpenAI states that current leading AI models require such material for effective training, as almost all human expressions are copyrighted. They argue restricting data to public domain sources would fail to meet today's needs. OpenAI acknowledges the need to support creators and has implemented measures to allow content blocking. However, the central issue is the unpaid use of copyrighted materials, with unresolved questions about licensing and associated costs impacting the generative AI industry. Read more at the-decoder.com.

📈 Trends

Emerging trends in 2024:

  1. Generative AI: AI continues to push creative boundaries, generating content that blurs the lines between human and machine creativity. Artists collaborate with AI as co-creators, ushering in unprecedented forms of artistic expression.

  2. Democratization of AI: AI accessibility is at an all-time high. ChatGPT, with its vast user base, exemplifies the ease of use, empowering individuals across various industries to harness AI's transformative power.

  3. Emergence of Multimodal AI: AI is becoming more multi-sensory, combining numeric data, text, images, and video to better understand context. This leads to more human-like interactions and fosters innovation across applications.

  4. Responsible AI: The ethical use of AI gains prominence in 2024. Global regulations are emerging to address concerns like manipulation, deepfakes, and bias, ensuring AI's responsible development and deployment.

  5. Quantum Computing and AI: Quantum computing's convergence with AI holds immense promise. Quantum AI is set to revolutionize complex problem-solving, unlocking outcomes beyond classical computing's reach.

  6. New AI Careers: The AI revolution isn't just about algorithms; it's about people. In 2024, diverse and exciting AI careers are on the rise, offering opportunities in AI ethics, project management, and more.

  7. Rise of Digital Twins: Digital twins, lifelike AI replicas, are becoming vital in various sectors, offering immersive experiences in virtual meetings, online shopping, and healthcare.

  8. Human-like Intelligence with Deep Learning: Deep learning approaches in AI are bringing us closer to human-like intelligence, enhancing accuracy across sectors like autonomous cars, OTT platforms, and e-commerce.

  9. AI Wearables: AI-infused wearables are reshaping the tech landscape. Devices like Humane AI's Pin and Tab seamlessly integrate into daily routines, enhancing digital experiences and possibilities.

  10. No-Code/Low-Code Programming: The no-code/low-code revolution continues to gain momentum in 2024. It enables non-programmers to drive innovation and efficiency across various domains, boosting productivity.

🗞️ COMPANIES

🏢 Google

Google Gemini: Everything you need to know about the new generative AI platform: Google's Gemini, a new generative AI platform, consists of three models: Gemini Ultra, Pro, and Nano, developed by DeepMind and Google Research. Unlike Google's text-only LaMDA, Gemini models handle multiple data types like text, audio, and images. Gemini Ultra is for complex tasks but hasn't been fully released yet. Gemini Pro is available and shows improved reasoning over models like GPT-3.5, accessible through Bard and Vertex AI. Gemini Nano, optimized for mobile devices, offers features like conversation summarization and smart replies on the Pixel 8 Pro. Despite its potential, Gemini faces early challenges and limitations in performance. Read more at TechcCrunch.com.

🗞️ IMPACT

💰 Business & Economy

Gen AI's potential to boost India's economy by $359 billion to $438 billion in 2029–30 is driven by sectors like business, finance, education, retail, and healthcare. Top Indian IT firms' adoption of Gen AI hinges on factors like readiness and infrastructure adaptability. Global clients are exploring AI outsourcing to enhance backend operations. Job dynamics in the IT sector are complex, with opportunities and restructuring. ChatGPT's role in deflating legacy services lies in its conversational abilities. Data security concerns exist for multinationals on Indian tech platforms, emphasizing the need for ethical data usage frameworks.

📚 LEARNING

🔬 Research

Memory, Consciousness and Large Language Model. (arXiv:2401.02509v1 [q-bio.NC]): With the development in cognitive science and Large Language Models (LLMs), increasing connections have come to light between these two distinct fields. Building upon these connections, we propose a conjecture suggesting the existence of a duality between LLMs and Tulving's theory of memory. We identify a potential correspondence between Tulving's synergistic ecphory model (SEM) of retrieval and the emergent abilities observed in LLMs, serving as supporting evidence for our conjecture.
Read more at arxiv.org.

Fine-tuning and Utilization Methods of Domain-specific LLMs. (arXiv:2401.02981v1 [cs.CL]): Recent releases of pre-trained Large Language Models (LLMs) have gained considerable traction, yet research on fine-tuning and employing domain-specific LLMs remains scarce. This study investigates approaches for fine-tuning and leveraging domain-specific LLMs, highlighting trends in LLMs, foundational models, and methods for domain-specific pre-training. Focusing on the financial sector, it details dataset selection, preprocessing, model choice, and considerations crucial for LLM fine-tuning in...
Read more at arxiv.org.

📚 TOOLING

🧬 Frameworks

Microsoft's research introduces WaveCoder, an efficient coding language model, trained with a novel dataset called CodeOcean. CodeOcean, comprising 20,000 diverse coding examples, enables fine-tuning of models like WaveCoder for specific coding tasks while being cost-effective. This approach balances dataset size with performance, achieving high efficiency in tasks such as code generation, summarization, language translation, and code repair. WaveCoder demonstrates superior performance compared to models trained on similar-sized datasets, particularly in code summarization and repair tasks. Microsoft is considering releasing WaveCoder and CodeOcean, with potential future research exploring larger datasets and integrations.

🧠 THOUGHTS

💬 Opinion

OpenAI's recent announcement allowing anyone with a $20 ChatGPT Plus subscription to create their own GPT models has sparked excitement among consumers and creators, leading to a variety of niche GPTs. However, this blog post outlines significant concerns for businesses, detailing nine key issues ranging from data privacy and content leakage to limited knowledge bases and deployment restrictions. These challenges present major hurdles for businesses looking to utilize these GPTs effectively, indicating a need for more robust solutions from OpenAI.

Key Points:

  1. Training on Your Data: Uploaded data can be incorporated into ChatGPT's knowledge base, risking confidential business information becoming public.

  2. Data Privacy: Competitors can potentially access and download uploaded files, compromising sensitive data.

  3. Leaking Custom Instructions: Custom instructions for GPT control can be leaked, enabling competitors to replicate or improve upon your GPT.

  4. Cannot Embed Into Websites: Personalized GPTs cannot be integrated into business websites, limiting their utility.

  5. Hallucination: GPTs may generate inaccurate or fabricated information, posing a risk for business reputation and legal issues.

  6. No Chat Logs: Lack of access to chat logs prevents businesses from auditing AI interactions and understanding customer behavior.

  7. Very Limited Knowledge: The knowledgebase is limited to 20 files and 1.5M words, which is insufficient for many businesses.

  8. Cannot Ingest Website Data: GPTs are unable to ingest website data directly, limiting their effectiveness as a comprehensive business tool.

  9. Only Accessible to Paid Subscribers: Restricting access to paid ChatGPT Plus subscribers hinders widespread deployment for customer and employee use.

Your Feedback

I want this newsletter to be valuable to you so if there's anything on your mind—praises, critiques, or just a hello—please drop me a note. You can hit reply or shoot me a message directly at my email address: [email protected].

Join my community by subscribing to my newsletter below:

Reply

or to participate.