Generative AI Weekly - Apr. 4, 2024

This week:

• Using ChatGPT for UX research - experimental results
• More competitions and benchmarks for AI progress - the AI Mathematical Olympiad "this competition uses a dataset of 110 novel math problems… from simple arithmetic to algebraic thinking and geometric reasoning"
• Podcast of Ethan Mollick and Erza Klein on How to Use AI right now. "Mollick says it’s helpful to understand this moment as one of co-creation, in which we all should be trying to make sense of what this technology is going to mean for us."
• Evidence for extensive appearance of chatGPT/LLM derived text in scholarly papers.. A paper finding signals and evidence of GPT use. Another one for medical papers here.
• Large language models, explained with a minimum of math and jargon
• "The 18 most interesting startups from YC’s Demo Day show we’re in an AI bubble"
• The 2024 MAD (ML, AI & Data) Landscape
• Biased source, but "16 Changes to the Way Enterprises Are Building and Buying Generative AI". I think the most useful indicator here to see is that the majority of use cases are internal facing (not customer facing) for now.
• In this paper on a new evaluation benchmark for financial questions, there is evidence that giving context before asking a question returns better results. "Showing the relevant context (i.e., filing or evidence extract) before the question leads to significant performance improvements over showing the context after the question."
• "In 1979, IBM—like all technology businesses—had a meeting about future technology, both hypothetical and real…. Now, a single photograph from this meeting's presentation packet has resurfaced after years of circulating the internet, ready for its moment nearly 45 years later."
• Fact-checking via LLM seems to outperform crowdsourced human annotators… 20x cheaper.