Skip to content

Latest commit

 

History

History
50 lines (33 loc) · 2.1 KB

index.md

File metadata and controls

50 lines (33 loc) · 2.1 KB

User Defined Functions in Python for Cosmian KMS

Cosmian maintains a Python UDF library that enables efficient, large-scale cryptographic operations with the Cosmian KMS.

Purpose and Applications

These User Defined Functions (UDFs) are designed specifically for:

  • Data processing platforms like PySpark, Databricks, Snowflake, and Kafka
  • Implementing application-level encryption in Big Data environments
  • Protecting sensitive data by encrypting it before storage and only decrypting during processing
  • Minimizing clear-text exposure of sensitive information

Technical Capabilities

The library provides:

  • Batch processing capabilities for efficient encryption/decryption
  • Parallelization to optimize throughput
  • Support for multiple encryption contexts and key identifiers
  • Performance optimizations for Big Data workloads

Expected throughput is in the range of 5 million decryption in 5 seconds with a 10 vCPU KMS instance.

Supported Cryptographic Algorithms

The UDFs support several NIST-approved and RFC-standardized algorithms:

  • AES-GCM (NIST SP 800-38D): Standard authenticated encryption
  • AES-GCM-SIV (RFC 8452): Deterministic authenticated encryption with nonce misuse resistance
  • AES-XTS (NIST SP 800-38E): Storage encryption for fixed-length data
  • ChaCha20-Poly1305 (RFC 8439): High-performance authenticated encryption

Performance Considerations

Key performance factors include:

  • Network latency between the UDF execution environment and KMS server
  • Batch size optimization for your specific workload
  • Algorithm selection based on security requirements
  • Potential use of deterministic encryption (like AES-GCM-SIV) for performance-critical applications

Check the encryption and decryption a scale section for details.

Getting Started

For implementation details, examples, and best practices, refer to the GitHub repository, particularly the tests directory which contains practical code samples for various use cases.