TLDR;
This podcast episode features Garrett Lord, the CEO of Handshake, discussing the company's expansion into AI data labeling. Handshake, originally a platform connecting college students with employers, has leveraged its extensive network of students and alumni to provide high-quality training data for AI labs. Garrett explains the shift in AI development from pre-training to post-training, where expert data is crucial for refining AI models. He highlights Handshake's unique advantage in accessing and engaging specialized talent, leading to rapid growth and significant revenue in the AI data sector. The conversation also touches on the impact of AI on job markets, the importance of continuous learning, and strategies for building successful businesses within existing companies.
- Handshake's transition into AI data labeling, leveraging its existing network of students and alumni.
- The importance of high-quality, expert-driven data for post-training AI model refinement.
- Strategies for building and scaling a new business within an established company.
Understanding Data Labeling and Post-Training in AI [5:04]
Garrett Lord explains that training AI models involves two main phases: pre-training and post-training. Pre-training focuses on feeding the model vast amounts of general data, like the entire internet, to establish a broad base of knowledge. However, gains from pre-training have plateaued as models have already absorbed most available data. Post-training involves refining the model with high-quality, specific data to improve capabilities in areas like coding, mathematics, law, and finance. This phase includes techniques like reinforcement learning with human feedback (RLHF) and fine-tuning, where experts provide data to correct flaws and improve reasoning.
Handshake's Unique Approach to Data Labeling [9:48]
Handshake's unique value proposition lies in its access to a large, engaged audience of professionals, including PhDs and master's students, across various academic disciplines. This allows Handshake to hyper-target experts and gather specialized data that hasn't been available on the internet before. Garrett emphasizes that the data labeling market has shifted from generalist tasks to requiring experts who can identify and correct flaws in AI models. These experts are crucial for improving models in economically valuable areas like STEM, accounting, law, medicine, and finance.
The Role of Experts and the Data Creation Process [13:08]
Garrett describes how experts, such as PhDs in biology or education, work to identify flaws in AI-generated content and provide correct answers to fine-tune the models. For example, a biology PhD might find errors in a model's biological explanations, while an education PhD might improve educational design models. These experts use tools provided by Handshake to interact with the latest models, create data, and ensure its quality. The data is often structured in JSON format and includes elements like step-by-step reasoning and rubrics for evaluation.
Quality, Volume, and Speed in Data Labeling [19:28]
Model builders prioritize three key factors: quality, volume, and speed. High-quality data is essential to avoid training models with incorrect information. Generating large volumes of data in advanced domains requires access to top experts from leading institutions. Speed is crucial because researchers are constantly testing hypotheses and need quick turnaround times to scale successful pipelines. Handshake focuses on assessing each unit of data, using post-training teams and GPUs to ensure quality and provide insights to researchers.
AI's Impact on Job Markets and the Advantage of AI Natives [24:18]
Garrett believes that AI will enhance human productivity and create new economic opportunities rather than eliminate jobs. He notes that younger people who have grown up with AI tools have a significant advantage in leveraging these technologies. These "AI natives" can accomplish more with AI-enabled tools, making them highly valuable in the workforce. While some jobs will evolve or become displaced, continuous learning and upskilling will be essential for workers to adapt.
Handshake's Transition into AI Data Labeling [33:06]
Handshake's move into AI data labeling was a natural extension of its mission to help people start and advance their careers. The company realized the value of its extensive network of experts when middleman companies began recruiting PhDs and master's students from the platform. By cutting out the middleman and directly serving the needs of both the experts and the AI labs, Handshake created a more efficient and rewarding experience. This new business quickly achieved significant revenue, demonstrating the immense demand for high-quality training data.
Building a New Business Within an Existing Company [45:48]
Garrett shares insights on the challenges and strategies for building a new business within an established company. Key elements include maintaining a separate team, focusing on a single customer initially, and empowering the new team with ownership and autonomy. He emphasizes the importance of a metrics-driven approach and a culture that celebrates impact. By creating a distinct identity and fostering a sense of urgency, Handshake successfully launched its AI data labeling business while continuing to grow its core platform.
The Future of AI and the Importance of Human Expertise [1:00:19]
Garrett discusses the future of AI, emphasizing that the type of data needed will continue to evolve, including CAD files, scientific tool use data, and multimodal data. While synthetic data may play a role, human expertise will remain crucial for ensuring the quality and relevance of training data. He encourages entrepreneurs to focus on creating meaningful solutions that improve people's lives, highlighting the vast opportunities in AI to enhance learning and job matching.