feat: add deploy grpc tts serve on modal

[weedge]

feat:

• add deploy grpc tts serve on modal

fix:

• pub achatbot pypi script and pyproject

f5 tts: #101

# download model and asset
modal run src/download_models.py --repo-ids "SWivid/F5-TTS"
modal run src/download_models.py --repo-ids "charactr/vocos-mel-24khz"
modal run src/download_assets.py --asset-urls "https://raw.githubusercontent.com/SWivid/F5-TTS/refs/heads/main/src/f5_tts/infer/examples/vocab.txt"
modal run src/download_assets.py --asset-urls "https://raw.githubusercontent.com/SWivid/F5-TTS/refs/heads/main/src/f5_tts/infer/examples/basic/basic_ref_en.wav" 
modal run src/download_assets.py --asset-urls "https://raw.githubusercontent.com/SWivid/F5-TTS/refs/heads/main/src/f5_tts/infer/examples/basic/basic_ref_zh.wav" 

# run tts grpc server
TTS_TAG=tts_f5 IMAGE_GPU=T4 modal run src/grpc_tts_serve.py

# run tts grpc client
SERVE_ADDR=r21.modal.host:45375 TTS_TAG=tts_f5 F5TTS_MODEL_CKPT_PATH=/root/models/SWivid/F5-TTS/F5TTS_Base/model_1200000.safetensors F5TTS_VOCODER_MODEL_DIR=/root/models/charactr/vocos-mel-24khz  IS_RELOAD=1 python -m src.cmd.grpc.speaker.client

spark tts: #130

# download model and asset
modal run src/download_models.py --repo-ids "SparkAudio/Spark-TTS-0.5B"

# run tts grpc server
TTS_TAG=tts_spark IMAGE_GPU=T4 modal run src/grpc_tts_serve.py

# run tts grpc client
SERVE_ADDR=r21.modal.host:36349 TTS_TAG=tts_spark TTS_MODEL_DIR=/root/.achatbot/models/SparkAudio/Spark-TTS-0.5B TTS_LM_MODEL_PATH=/root/.achatbot/models/SparkAudio/Spark-TTS-0.5B/LLM  IS_RELOAD=1 python -m src.cmd.grpc.speaker.client

orpheus tts: #133

# download model and asset
modal run src/download_models.py --repo-ids "canopylabs/orpheus-3b-0.1-ft"
modal run src/download_models.py --repo-ids "hubertsiuzdak/snac_24khz"

# run tts grpc server
TTS_TAG=tts_orpheus IMAGE_GPU=T4 modal run src/grpc_tts_serve.py

# run tts grpc client
SERVE_ADDR=r21.modal.host:45513 TTS_TAG=tts_orpheus LM_MODEL_PATH=/root/.achatbot/models/canopylabs/orpheus-3b-0.1-ft CODEC_MODEL_PATH=/root/.achatbot/models/hubertsiuzdak/snac_24khz  IS_RELOAD=1 python -m src.cmd.grpc.speaker.client

[gemini-code-assist]

Hello @weedge, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

Summary of Changes

This pull request focuses on adding the capability to deploy gRPC-based TTS (Text-to-Speech) services on Modal, along with some fixes to the achatbot pypi script and pyproject.toml. It includes changes to download assets and models, and introduces a new script for serving TTS via gRPC on Modal.

Highlights

• gRPC TTS Deployment on Modal: Adds a new grpc_tts_serve.py script to deploy TTS services on Modal using gRPC. This includes setting up a Modal app, defining an image with necessary dependencies, and running the TTS server within a Modal function.
• Model and Asset Downloading: Updates download_models.py and download_assets.py to support downloading models and assets required for the new TTS services. Includes example commands for downloading specific models like F5-TTS, Spark-TTS, and Orpheus TTS.
• Pyproject.toml Updates: Modifies pyproject.toml to update the achatbot version and add dependencies for different TTS models (F5, Spark, Orpheus).
• Pypi Script Fixes: Fixes the pypi_achatbot.sh script to correctly replace src. and deps. imports with achatbot. imports when building the pypi package.

Changelog

Click here to see the changelog

• deploy/modal/src/download_assets.py
  • Modified the logic for extracting filenames from URLs to handle cases where the '/assets/' path is not present.
  • Added example commands for downloading assets for F5-TTS.
• deploy/modal/src/download_models.py
  • Added secrets=[modal.Secret.from_name("achatbot")] to the modal function definition.
  • Added print statement to indicate model downloading progress.
  • Added max_workers=8 to the snapshot_download function call.
  • Added example commands for downloading models for F5-TTS, Spark-TTS and Orpheus TTS.
• deploy/modal/src/grpc_tts_serve.py
  • Created a new script to deploy gRPC TTS services on Modal.
  • Defines a Modal app, sets up a Docker image with necessary dependencies (achatbot, grpc, ffmpeg, etc.).
  • Creates a Modal function to run the TTS server, forwarding port 50052 for gRPC communication.
  • Includes example commands for running the TTS server with different TTS models (tts_f5, tts_spark, tts_orpheus).
• pyproject.toml
  • Updated the achatbot version from 0.0.8.13 to 0.0.9.
  • Added dependencies for F5-TTS, including accelerate, bitsandbytes, tomli, cached_path, click, matplotlib, and numpy<=1.26.4.
  • Added specific versions for numpy, torch, torchaudio, and transformers for orpheus tts.
• scripts/pypi_achatbot.sh
  • Modified the script to correctly replace from src. and from deps. imports with from achatbot. imports.
  • Added replacement rules for import src. and import deps. statements.
• src/cmd/grpc/speaker/client.py
  • Replaced host and port environment variables with SERVE_ADDR to define the gRPC server address.
• src/types/llm/transformers.py
  • Removed unused import torch statement.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in issue comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

---

Did you know?

The first speech synthesizer was invented in 1779 by Christian Kratzenstein, a professor of physiology at the University of Copenhagen.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

The pull request introduces gRPC TTS serving capabilities to the achatbot project, along with related updates and fixes. The changes include adding a new grpc_tts_serve.py file, modifying download_assets.py and download_models.py, updating pyproject.toml to include new dependencies, fixing a script for pypi package, and adjusting the gRPC client. Overall, the changes seem well-structured and aim to enhance the project's TTS functionality.

Summary of Findings

• Missing Error Handling in download_assets.py: The download_assets function in download_assets.py could benefit from more robust error handling, particularly around network issues or file system errors. Consider adding retry mechanisms or more informative error messages to improve the user experience and debuggability.
• Dependency Versioning in pyproject.toml: The pyproject.toml file includes several dependency updates and additions. It's crucial to ensure that these versions are compatible with each other and the rest of the project. Consider using a dependency management tool to automatically resolve conflicts and ensure consistency.
• Inconsistent use of environment variables: The code uses environment variables for configuration, but the usage is not always consistent. For example, some variables have default values while others don't. Consider standardizing the way environment variables are used to improve maintainability and reduce the risk of misconfiguration.

Merge Readiness

The pull request introduces significant new functionality and includes several important fixes. However, there are some areas that could benefit from further attention, particularly around error handling and dependency management. I recommend addressing the identified issues before merging to ensure the stability and maintainability of the project. I am unable to approve this pull request, and recommend that others review and approve this code before merging.