Shop all Manning Publications

Build a Large Language Model (from Scratch) - (From Scratch) by Sebastian Raschka (Paperback)

Name: Build a Large Language Model (from Scratch) - (From Scratch) by Sebastian Raschka (Paperback)
Brand: Manning Publications
SKU: 92204464
Price: 49.24 USD
Availability: InStock

$49.24Save $10.75 (18% off)

In Stock

Free & easy returns

Return this item by mail or in store within 90 days for a full refund.

Eligible for registries and wish lists

Save 5% every day

With Target Circle™ Card (restrictions apply)

About this item

Details

Highlights

How to implement LLM attention mechanisms and GPT-style transformers.
About the Author: Sebastian Raschka is a Staff Research Engineer at Lightning AI, where he works on LLM research and develops open-source software.
368 Pages
Computers, Artificial Intelligence
Series Name: From Scratch

Description

About the Book

Learn how to create, train, and tweak large language models (LLMs) by building one from the ground up!

Book Synopsis

How to implement LLM attention mechanisms and GPT-style transformers.

In Build a Large Language Model (from Scratch) bestselling author Sebastian Raschka guides you step by step through creating your own LLM. Each stage is explained with clear text, diagrams, and examples. You'll go from the initial design and creation, to pretraining on a general corpus, and on to fine-tuning for specific tasks.

Build a Large Language Model (from Scratch) teaches you how to:

Plan and code all the parts of an LLM
Prepare a dataset suitable for LLM training
Fine-tune LLMs for text classification and with your own data
Use human feedback to ensure your LLM follows instructions
Load pretrained weights into an LLM

Build a Large Language Model (from Scratch) takes you inside the AI black box to tinker with the internal systems that power generative AI. As you work through each key stage of LLM creation, you'll develop an in-depth understanding of how LLMs work, their limitations, and their customization methods. Your LLM can be developed on an ordinary laptop, and used as your own personal assistant.

About the technology

Physicist Richard P. Feynman reportedly said, "I don't understand anything I can't build." Based on this same powerful principle, bestselling author Sebastian Raschka guides you step by step as you build a GPT-style LLM that you can run on your laptop. This is an engaging book that covers each stage of the process, from planning and coding to training and fine-tuning.

About the book

Build a Large Language Model (From Scratch) is a practical and eminently-satisfying hands-on journey into the foundations of generative AI. Without relying on any existing LLM libraries, you'll code a base model, evolve it into a text classifier, and ultimately create a chatbot that can follow your conversational instructions. And you'll really understand it because you built it yourself!

What's inside

Plan and code an LLM comparable to GPT-2
Load pretrained weights
Construct a complete training pipeline
Fine-tune your LLM for text classification
Develop LLMs that follow human instructions

About the reader

Readers need intermediate Python skills and some knowledge of machine learning. The LLM you create will run on any modern laptop and can optionally utilize GPUs.

About the author

Sebastian Raschka is a Staff Research Engineer at Lightning AI, where he works on LLM research and develops open-source software.

The technical editor on this book was David Caswell.

Table of Contents

1 Understanding large language models
2 Working with text data
3 Coding attention mechanisms
4 Implementing a GPT model from scratch to generate text
5 Pretraining on unlabeled data
6 Fine-tuning for classification
7 Fine-tuning to follow instructions
A Introduction to PyTorch
B References and further reading
C Exercise solutions
D Adding bells and whistles to the training loop
E Parameter-efficient fine-tuning with LoRA

From the Back Cover

From the back cover:

About the reader:

Readers need intermediate Python skills and some knowledge of machine learning. The LLM you create will run on any modern laptop and can optionally utilize GPUs.

Review Quotes

A fantastic resource for diving into LLMs--a must-read for anyone eager to get hands-on! --Dr. Vahid Mirjalili, Senior Data Scientist, FM Global

A clear, hands-on guide that empowers readers to build their own models and explore the cutting edge of AI.--Guillermo Alcántara, Project manager, PepsiCo Global

Must-have resource for quickly getting up to speed on LLMs. Whether you're new to the field or looking to deepen your knowledge, it's the perfect guide. --Walter Reade, Staff Developer Relations Engineer, Kaggle/Google

The most comprehensive book I've seen on building LLMs. Highly recommended! --Raul Ciotescu, CTO, Netzinkubator Software

About the Author

Sebastian Raschka is a Staff Research Engineer at Lightning AI, where he works on LLM research and develops open-source software.

The technical editor on this book was David Caswell.

Specifications

Dimensions (Overall): 9.2 Inches (H) x 7.4 Inches (W) x .9 Inches (D)

Weight: 1.35 Pounds

Suggested Age: 22 Years and Up

Number of Pages: 368

Genre: Computers

Sub-Genre: Artificial Intelligence

Series Title: From Scratch

Publisher: Manning Publications

Theme: Expert Systems

Format: Paperback

Author: Sebastian Raschka

Language: English

Street Date: October 29, 2024

TCIN: 92204464

UPC: 9781633437166

Item Number (DPCI): 247-26-9174

Origin: Made in the USA or Imported

If the item details aren’t accurate or complete, we want to know about it.

Shipping & Returns

Shipping details

Estimated ship dimensions: 0.9 inches length x 7.4 inches width x 9.2 inches height

Estimated ship weight: 1.35 pounds

We regret that this item cannot be shipped to PO Boxes.

This item cannot be shipped to the following locations: American Samoa (see also separate entry under AS), Guam (see also separate entry under GU), Northern Mariana Islands, Puerto Rico (see also separate entry under PR), United States Minor Outlying Islands, Virgin Islands, U.S., APO/FPO, Alaska, Hawaii

Return details

This item can be returned to any Target store or Target.com.

This item must be returned within 90 days of the date it was purchased in store, delivered to the guest, delivered by a Shipt shopper, or picked up by the guest.

See the return policy for complete information.