LLMs with MATLAB

Book available on Amazon 📖

https://www.amazon.com/dp/B0FJ2BRXXB

A practical guide to building Generative AI applications

Inspired from programming-GPTs, and based on the repo LLMs-with-MATLAB

This repo has been developed with MATLAB 24b and Python 3.10.

The code can be executed in MATLAB Online and MATLAB Desktop.

Some parts leverage other dev environments like VSCode and Jupyter.

Table of Contents

  Preface

  Chap 1 - Introducing LLMs

  Chap 2 - OpenAI APIs

  Chap 3 - the Art of Prompting

  Chap 4 - Question Answering

  Chap 5 - Agents and Tools

  Chap 6 - Speech to text and Text to speech

  Chap 7 - Vision

  Chap 8 - Image Generation

  Chap 9 - Appendix

  Utils

Preface

Connect to the official MathWorks repo matlab-deep-learning/llms-with-matlab

Setup your environment with dependencies and API keys.

addpath preface\
setup
edit preface\setup.mlx

Chap 1 - Introducing LLMs

Pay attention to the way LLMs learn.

addpath chap1_Introducing_LLMs\
attention

Chap 2 - OpenAI APIs

Make your first query in MATLAB.

edit chap2_OpenAI_APIs\chap2_1_first_query.m

Learn to manage messages history.

edit chap2_OpenAI_APIs\chap2_1_messages_history.mlx

Build your first chatbot, inspired by toshiakit/MatGPT

Chap 3 - the Art of Prompting

Chain, transform, organize, summarize to process large amount of information with LLMs. Learn all the methods.

Build a tidy bookmarking system.

edit chap3_Art_of_prompting\chap3_1_extract_content.m
edit chap3_Art_of_prompting\chap3_2_translate_articles.m
edit chap3_Art_of_prompting\chap3_3_classify_articles.m
edit chap3_Art_of_prompting\chap3_4_summarize_articles.m
edit chap3_Art_of_prompting\chap3_5_scale_processing.m

Chap 4 - Question Answering

Retrieve relevant information to augment the response of LLMs.

edit chap4_Question_answering\chap4_rag.mlx
edit chap4_Question_answering\chap4_embeddings.mlx

Chap 5 - Agents and Tools

Choose the right tool for the job. Think about it in steps.

edit chap5_Agents\chap5_OpenAI_functions.mlx

Chap 6 - Speech to text and Text to speech

Transcribe and synthetize information as you go.

Generate a daily tech podcast from your favorite website.

edit chap6_stt_tts\chap6_stt.mlx

Chap 7 - Vision

Analyze images and videos, detect objects.

Build an Optical Character Recognition app.

edit chap7_Vision\Chap7_Vision.mlx
edit chap7_Vision\Chap7_Detecting_cars.mlx
edit chap7_Vision\Chap7_OCR.mlx
edit chap7_Vision\Chap7_Video_understanding.mlx

Chap 8 - Image Generation

Create, edit, vary. Let your creativity run loose.

edit chap8_Image_generation\chap8_Image_generation.mlx

Chap 9 - Appendix

Learn more about AI, machine learning, deep learning, natural language processing and transformers.

Run open-source LLMs locally with Ollama.

edit chap9_appendix\Chap9_Ollama.mlx

Utils

export README.mlx README.md;