Issues

• Add grpc and graphql endpoints

  geniusrise/text#17 · Filter by author ixaxaar opened on Mar 26, 2024

• Integrate TorchDistributor based fine-tuning and inference on spark

  enhancement
  geniusrise/text#14 · Filter by author ixaxaar opened on Mar 1, 2024

• Support OAUTH token introspection in APIs

  enhancementgood first issue
  geniusrise/text#13 · Filter by author ixaxaar opened on Mar 1, 2024

• Add K-V cache, more options for compilation

  enhancementgood first issue
  geniusrise/text#12 · Filter by author ixaxaar opened on Mar 1, 2024

• Think about supporting triton - as triton server client or triton as lib + FFI

  question
  geniusrise/text#9 · Filter by author ixaxaar opened on Feb 27, 2024

• Add support for local models for llama.cpp in text

  enhancementgood first issue
  geniusrise/text#8 · Filter by author ixaxaar opened on Feb 27, 2024
  3

• Abstract out inference and batch inference - text

  good first issue
  geniusrise/text#7 · Filter by author ixaxaar opened on Feb 27, 2024

• Implement batching for APIs - text

  enhancement
  geniusrise/text#6 · Filter by author ixaxaar opened on Feb 27, 2024

• Implement streaming inference - text

  enhancement
  geniusrise/text#5 · Filter by author ixaxaar opened on Feb 27, 2024