Natural Language Processing in Action (2nd ed.)

Description

This seems to be a good, more recent alternative to Natural Language Processing With PyTorch (2019), though it's not PyTorch-specific.

🤗 Hugging Face AquinasLatinEnglish parallel corpus README.md: Byte-Pair Encoding (BPE) is far superior to word tokenization; cf. §2.2 "Beyond word tokens" (ref:13.38-73).

NMT: §9.2.3 "Transformer translation" (ref:22.53-114).

9.1.2 A LEGO set for language

04/08/269.1.2 A LEGO set for language : 769

Andrej Karpathy’s Jupyter Notebooks (minGPT and nanoGPT) that distill transformers down to their essence.9

9 The latest Andrej Karpathy notebook for learning and teaching about transformers is called nanoGPT (https://github.com/karpathy/nanoGPT).

04/08/269.1.2 A LEGO set for language : 778

When all is said and done, the deepest of the deep learning models are nothing more than a clever stacking of what is essentially linear and logistic regressions.

i.e., curve-fitting, as Briggs and Grant Sanderson, 3Blue1Brown: Linear regression is "the simplest form of machine learning": Transformers: The Tech Behind LLMs @8:19 said.

9.2.2 Connecting all the pieces

04/09/269.2.2 Connecting all the pieces : 784

For a mind-expanding walk through the modern GPT architecture, check out the 3Blue1Brown visualizations and explanations by Grant Sanderson.18

18 See Grant Sanderson’s animated tour of the GPT architecture (https://www.3blue1brown.com/lessons/gpt).

9.2.3 Transformer translation

04/09/269.2.3 Transformer translation : 805

TranslationTransformer complete model definition

04/09/269.2.3 Transformer translation : 814

Ben Trevett contributed much of the code for the PyTorch Transformer beginner tutorial. He, along with colleagues, has written an outstanding and informative Jupyter Notebook series for their PyTorch Seq2Seq tutorial,25 covering sequence-to-sequence models. Their “Attention Is All You Need” notebook26 provides a from-scratch implementation of a basic transformer model.

25 See “PyTorch Seq2Seq,” Ben Trevett (https://github.com/bentrevett/pytorch-seq2seq/).

26 See “Attention Is All You Need Jupyter Notebook,” Ben Trevett (https://github.com/bentrevett/pytorch-seq2seq/blob/main/legacy/6%20-%20Attention%20is%20All%20You%20Need.ipynb).

Develop your NLP skills from scratch, with an open source toolbox of Python packages, Transformers, Hugging Face, vector databases, and your own Large Language Models.

Natural Language Processing in Action, Second Edition has helped thousands of data scientists build machines that understand human language. In this new and revised edition, you’ll discover state-of-the art Natural Language Processing (NLP) models like BERT and HuggingFace transformers, popular open-source frameworks for chatbots, and more. You’ll create NLP tools that can detect fake news, filter spam, deliver exceptional search results and even build truthfulness and reasoning into Large Language Models (LLMs). In Natural Language Processing in Action, Second Edition you will learn how to: • Process, analyze, understand, and generate natural language text • Build production-quality NLP pipelines with spaCy • Build neural networks for NLP using Pytorch • BERT and GPT transformers for English composition, writing code, and even organizing your thoughts • Create chatbots and other conversational AI agents In this new and revised edition, you’ll discover state-of-the art NLP models like BERT and HuggingFace transformers, popular open-source frameworks for chatbots, and more. Plus, you’ll discover vital skills and techniques for optimizing LLMs including conversational design, and automating the “trial and error” of LLM interactions for effective and accurate results. Purchase of the print book includes a free eBook in PDF, Kindle, and ePub formats from Manning Publications. About the technology From nearly human chatbots to ultra-personalized business reports to AI-generated email, news stories, and novels, natural language processing (NLP) has never been more powerful! Groundbreaking advances in deep learning have made high-quality open source models and powerful NLP tools like spaCy and PyTorch widely available and ready for production applications. This book is your entrance ticket—and backstage pass—into the next generation of natural language processing. About the book Natural Language Processing in Action, Second Edition introduces the foundational technologies and state-of-the-art tools you’ll need to write and publish NLP applications. You learn how to create custom models for search, translation, writing assistants, and more, without relying on big commercial foundation models. This fully updated second edition includes coverage of BERT, Hugging Face transformers, fine-tuning large language models, and more. What's inside • NLP pipelines with spaCy • Neural networks with PyTorch • BERT and GPT transformers • Conversational design for chatbots About the reader For intermediate Python programmers familiar with deep learning basics. About the author Hobson Lane is a data scientist and machine learning engineer with over twenty years of experience building autonomous systems and NLP pipelines. Maria Dyshel is a social entrepreneur and artificial intelligence expert, and the CEO and cofounder of Tangible AI. Cole Howard and Hannes Max Hapke were co-authors of the first edition. Table fo Contents Part 1 1 Machines that read and write: A natural language processing overview 2 Tokens of thought: Natural language words 3 Math with words: Term frequency–inverse document frequency vectors 4 Finding meaning in word counts: Semantic analysis Part 2 5 Word brain: Neural networks 6 Reasoning with word embeddings 7 Finding kernels of knowledge in text with CNNs 8 Reduce, reuse, and recycle your words: RNNs and LSTMs Part 3 9 Stackable deep learning: Transformers 10 Large language models in the real world 11 Information extraction and knowledge graphs 12 Getting chatty with dialog engines A Your NLP tools B Playful Python and regular expressions C Vectors and linear algebra D Machine learning tools and techniques E Deploying NLU containerized microservices F Glossary