October 14, 2025
Workshop #1 provides a practical overview of fine-tuning large language models, focusing on when it is and is not beneficial, emphasizing a workflow of simplification, prototyping, and iterative improvement using evaluations.
Workshop #2 builds on Workshop 1 to focus on practical fine-tuning of LLMs, covering model selection, fine-tuning techniques with Axolotl, data quality improvement, debugging, and using tools like Accelerate and Modal.
This talk by Kyle Corbitt from OpenPipe outlines ten key recommendations for successfully deploying fine-tuned language models (LLMs) in production.
This Q&A session covered various topics, including template-free prompt construction, data type selection for HuggingFace datasets, DPO and RLHF, understanding chat templates and dataset types, ensuring consistent tokenization, multimodal fine-tuning, and future directions for Axolotl.
This Q&A session covers various aspects of LLM fine-tuning, including tools, techniques, data sets, and hardware considerations.
Freddy showcases Gradio’s features, advantages, and repository contributions, highlighting its potential for AI applications. He concludes with insights into its future roadmap, which includes enhanced agent workflows, real-time streaming, and improved UI features.
Workshop #3 focuses on the crucial role of evaluation in fine-tuning and improving LLMs. It covers three main types of evaluations: unit tests, LLM as a judge, and human evaluation.
This talk by Ankur Goyal from BrainTrust covers how to build evals for LLM systems by walking through a Text2SQL use case.
This talk by John Berryman covers the fundamentals of language models, prompt engineering techniques, and building LLM applications.
In this talk, J.J. Allaire walks through the core concepts and design of the Inspect framework and demonstrate its use for a variety of evaluation tasks.
This Q&A session covers a wide array of topics related to Modal, a platform designed to simplify the execution of Python code in the cloud.
This Q&A session on LangChain/LangSmith covers topics like product differentiation, features, use cases, agent workflows, data set creation, and full-stack development for ML engineers.
In this talk, Jonathan Whitaker from answer.ai shows how to build intuition around training performance with a focus on GPU-poor fine tuning.
This Q&A session covers a wide range of topics related to LLMs, including practical tips for training and optimization, insights into the current research landscape, and thoughts on future trends.
In this talk, Abhishek Thakur, who leads AutoTrain at 🤗, shows how to use 🤗 AutoTrain to train/fine-tune LLMs without having to write any code.
Workshop #4 focuses on the practical aspects of deploying fine-tuned LLMs, covering various deployment patterns, performance optimization techniques, and platform considerations.
In this talk, Sophia Yang from Mistal AI covers best practices for fine-tuning Mistral language models. It covers Mistral’s capabilities, the benefits of fine-tuning over prompting, and provides practical demos using the Mistral Fine-tuning API and open-source codebase.
In this talk, Daniel van Strien from 🤗 outlines key considerations and techniques for creating high-quality datasets for fine-tuning LLMs.
In this talk, Emmanuel Ameisen from Anthropic argues that fine-tuning LLMs is often less effective and efficient than focusing on fundamentals like data quality, prompting, and Retrieval Augmentation Generation (RAG).
In this talk, Jason Liu covers a a systematic approach to improving Retrieval Augmented Generation (RAG) applications.
In this talk, Mark Saroufim and Jane Xu, discuss techniques and tools for mitigating Out of Memory (OOM) errors in PyTorch, specifically when working with LLMs.
In this talk, Paige Bailey, Generative AI Developer Relations lead at Google, discusses Google’s AI landscape with a focus on Gemini models and their applications.
In this talk, Ben Clavié from Answer.ai deconstructs the concept of Retrieval-Augmented Generation (RAG) and walks through building a robust, basic RAG pipeline.
In this talk, Charles Frye provides a deeper dive into Modal, exploring its capabilities beyond fine-tuning LLMs and demonstrating how it empowers users to build and deploy scalable, cost-efficient, and serverless applications with simplicity using Python.
This Q&A session on the Replicate platform covers topics like enterprise readiness, model deployment, application layers for LLMs, data privacy, logging, and potential future features.
In this talk, Hailey Schoelkopf from Eleuther AI provides an overview of the challenges in LLM evaluation, exploring different measurement techniques, highlighting reproducibility issues, and advocating for best practices like sharing evaluation code and using task-specific downstream evaluations.
In this talk, Simon Willison showcases LLM, a command-line tool for interacting with large language models, including how to leverage its plugin system, local model support, embedding capabilities, and integration with other Unix tools for tasks like retrieval augmented generation.
In this talk, Steven Heidel from OpenAI’s fine-tuning team covers best practices, use cases, and recent updates for fine-tuning OpenAI models.
This Q&A session with Predibase compares and contrasts Lorax, an open-source adapter-tuning library for large language models, with other similar libraries, highlighting its performance optimizations, unique features like dynamic adapter loading and support for various adapter types, and its role in a broader machine learning infrastructure strategy.
In this talk, Pawell Garbacki from Fireworks.ai covers the process and best practices of finetuning an LLM for function/tool use.
In this talk, Jo Kristian Bergum from Vespa.ai explores practical strategies for building better Retrieval Augmented Generation (RAG) applications, emphasizing the importance of robust evaluation methods and understanding the nuances of information retrieval beyond simple vector embeddings.
This live discussion between six AI experts and practitioners centers on the practical lessons learned from a year of building real-world applications with LLMs, emphasizing the critical importance of data literacy, rigorous evaluation, and iterative development processes.