Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 2 additions & 13 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -65,19 +65,6 @@ __pycache__/
!e.*
!libkurl.*
!libobjstor.*
**/4-1-libs/*.q
**/4-1-libs/*.q_
**/4-1-libs/*.k_
**/4-1-libs/*.k
!**/4-1-libs/q.k
**/4-1-libs/**/libe.so
**/4-1-libs/**/e.*

#ToDo revert in KXI-44621
src/pykx/lib/4-1-libs/l64/kxreaper
src/pykx/lib/4-1-libs/l64/libkurl.so
src/pykx/lib/4-1-libs/l64/libobjstor.so
src/pykx/lib/4-1-libs/l64/pg

# Distribution / packaging
.Python
Expand Down Expand Up @@ -252,3 +239,5 @@ symsEnumsSplay/**

src/pykx/pykx_init.q_
qcumber_results.xml

src/pykx/lib/4-1-libs
10 changes: 5 additions & 5 deletions DEVELOPING.md
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
# Development Guidelines

This document is for developers of PyKX, and strives to clarify some otherwise unspoken assumptions regarding PyKX, and to provide some guidelines for its development.
This document is for developers of KDB-X Python, and strives to clarify some otherwise unspoken assumptions regarding KDB-X Python, and to provide some guidelines for its development.


## Goals

PyKX aims to provide a Python-first interface to q. This means that the API should be as Pythonic as possible while ideally still being able to efficiently perform any task that one could be done with q itself.
KDB-X Python aims to provide a Python-first interface to q. This means that the API should be as Pythonic as possible while ideally still being able to efficiently perform any task that one could be done with q itself.

The primary target audience of this package is naive Python users. These would be data scientists, and the majority of programmers who have at least a passing familiarity with Python. These users will rely heavily on the documentation, and examples. They will not be familiar with any advanced Python concepts, or even have a working knowledge of the [Python object model](https://docs.python.org/3/reference/datamodel.html). Additionally, and more importantly for any developers of PyKX who are primarily q programmers, these users will not be familiar with any q-isms, or vector languages in general. The closest thing most of the will have experienced to a vector language is the Python package [Numpy](https://numpy.org/), and as such we can expect that many of them will use PyKX primarily as a way to convert query results from their q database to Numpy arrays.
The primary target audience of this package is naive Python users. These would be data scientists, and the majority of programmers who have at least a passing familiarity with Python. These users will rely heavily on the documentation, and examples. They will not be familiar with any advanced Python concepts, or even have a working knowledge of the [Python object model](https://docs.python.org/3/reference/datamodel.html). Additionally, and more importantly for any developers of KDB-X Python who are primarily q programmers, these users will not be familiar with any q-isms, or vector languages in general. The closest thing most of the will have experienced to a vector language is the Python package [Numpy](https://numpy.org/), and as such we can expect that many of them will use KDB-X Python primarily as a way to convert query results from their q database to Numpy arrays.

Other kinds of users who should also be considered include skilled q users, and skilled Python users. For them we try to include extra opt-in features which are entirely unnecessary to get the core experience of PyKX. For example the q console is, in part, provided for skilled q users, and it does not follow any Python conventions.
Other kinds of users who should also be considered include skilled q users, and skilled Python users. For them we try to include extra opt-in features which are entirely unnecessary to get the core experience of KDB-X Python. For example the q console is, in part, provided for skilled q users, and it does not follow any Python conventions.


## What does Pythonic means?
Expand Down Expand Up @@ -74,7 +74,7 @@ Other miscellaneous Pythonic tips include:

## API Development Considerations

It's easy to provide new functions/methods, decide on argument order, calling conventions, etc., but it's terribly difficult to change those decisions after PyKX has been released. For this reason we should err on the side of not releasing such changes until we're confident that we have made the best decision, and that we probably won't want to change it in the future.
It's easy to provide new functions/methods, decide on argument order, calling conventions, etc., but it's terribly difficult to change those decisions after KDB-X Python has been released. For this reason we should err on the side of not releasing such changes until we're confident that we have made the best decision, and that we probably won't want to change it in the future.


## Cython Tips
Expand Down
6 changes: 3 additions & 3 deletions LICENSE.txt
Original file line number Diff line number Diff line change
Expand Up @@ -218,8 +218,8 @@ Software License for q.so

KX Confidential

Version Number: 1.0
Date Last Revised: April 2023
Version Number: 1.1
Date Last Revised: June 2026

Software License Agreement for use of q.so (“Agreement”)

Expand All @@ -231,7 +231,7 @@ This Agreement is made between KX Systems, Inc. (“KX” or “we”) and the E

 1. LICENSE GRANTS

1.1 Grant of License. KX hereby grants End User a non-transferable, non-exclusive license, without right of sublicense, to install and use the Software solely for the purpose of compiling and running the PyKX software made available by KX under separate licensing terms. End User will not attempt to circumvent any restrictions imposed on the Software or use the Software for any purpose other than stated above. If End User is using the PyKX software under the terms of a commercial licence from KX, the End User must obtain a separate licence from KX for the use of the Software.
1.1 Grant of License. KX hereby grants End User a non-transferable, non-exclusive license, without right of sublicense, to install and use the Software solely for the purpose of compiling and running the PyKX or KDB-X software made available by KX under separate licensing terms. End User will not attempt to circumvent any restrictions imposed on the Software or use the Software for any purpose other than stated above. If End User is using the PyKX or KDB-X software under the terms of a commercial licence from KX, the End User must obtain a separate licence from KX for the use of the Software.

1.2 Software Use Restrictions. End User may not: (a) modify any part of the Software or create derivative works thereof, (b) sell, lease, license or distribute the Software to any third party, (c) attempt to decompile, disassemble or reverse engineer the Software, (d) copy the Software, except for purposes of installing and executing it within the limitations set out at clause 1.1, (e) use or attempt to use the Software in any way that is unlawful or fraudulent or has any unlawful or fraudulent purpose or effect, (f) use or attempt to use the Software in any way that would breach the license granted herein.

Expand Down
71 changes: 26 additions & 45 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,20 +1,20 @@
# PyKX
# KDB-X Python

## Introduction

PyKX is a Python first interface to the worlds fastest time-series database kdb+ and it's underlying vector programming language q. PyKX takes a Python first approach to integrating q/kdb+ with Python following 10+ years of integrations between these two languages. Fundamentally it provides users with the ability to efficiently query and analyze huge amounts of in-memory and on-disk time-series data.
KDB-X Python is a Python first interface to the worlds fastest time-series database kdb+ and it's underlying vector programming language q. KDB-X Python takes a Python first approach to integrating q/kdb+ with Python following 10+ years of integrations between these two languages. Fundamentally it provides users with the ability to efficiently query and analyze huge amounts of in-memory and on-disk time-series data.

This interface exposes q as a domain-specific language (DSL) embedded within Python, taking the approach that q should principally be used for data processing and management of databases. This approach does not diminish the ability for users familiar with q or those wishing to learn more about it from making the most of advanced analytics and database management functionality but rather empowers those who want to make use of the power of kdb+/q who lack this expertise to get up and running fast.

PyKX supports three principal use cases:
KDB-X Python supports three principal use cases:

- It allows users to store, query, manipulate and use q objects within a Python process.
- It allows users to query external q processes via an IPC interface.
- It allows users to embed Python functionality within a native q session using it's under q functionality.

Users wishing to install the library can do so following the instructions [here](https://code.kx.com/pykx/getting-started/installing.html).

Once you have the library installed you can get up and running with PyKX following the quickstart guide [here](https://code.kx.com/pykx/getting-started/quickstart.html).
Once you have the library installed you can get up and running with KDB-X Python following the quickstart guide [here](https://code.kx.com/pykx/getting-started/quickstart.html).

### What is q/kdb+?

Expand All @@ -31,70 +31,51 @@ For more information on using q/kdb+ and getting started with see the following

## Installation

### Installing PyKX using `pip`
### Installing KDB-X Python using `pip`

Ensure you have a recent version of pip:

```bash
pip install --upgrade pip
```

Then install the latest version of PyKX with the following command:
Then install the latest version of KDB-X Python with the following command:

```bash
pip install pykx
pip install --upgrade pykx
```

To install a specific version of PyKX run the following command replacing <INSERT_VERSION> with a specific released semver version of the interface
To install a specific version of KDB-X Python run the following command replacing <INSERT_VERSION> with a specific released semver version of the interface

```bash
pip install pykx==<INSERT_VERSION>
```

**Warning:** Python packages should typically be installed in a virtual environment. [This can be done with the venv package from the standard library](https://docs.python.org/3/library/venv.html).

### PyKX License access and enablement
### KDB-X Python License access and enablement

Installation of PyKX via pip provides users with access to the library with limited functional scope, full details of these limitations can be found [here](docs/user-guide/advanced/modes.md). To access the full functionality of PyKX you must first download and install a kdb+ license, this can be achieved either through use of a personal evaluation license or receipt of a commercial license.
Installation of KDB-X Python via pip provides users with access to the library with limited functional scope, full details of these limitations can be found [here](docs/user-guide/advanced/modes.md). To access the full functionality of KDB-X Python you must first download and install a license.

#### Personal Evaluation License
If you do not have a license go to the [KX Developer Center](https://developer.kx.com/products/kdb-x/install) to obtain a KDB-X license.

The following steps outline the process by which a user can gain access to an install a kdb Insights license which provides access to PyKX

1. Visit https://kx.com/kdb-insights-sdk-personal-edition-download/ and fill in the attached form following the instructions provided.
2. On receipt of an email from KX providing access to your license download this file and save to a secure location on your computer.
3. Set an environment variable on your computer pointing to the folder containing the license file (instructions for setting environment variables on PyKX supported operating systems can be found [here](https://chlee.co/how-to-setup-environment-variables-for-windows-mac-and-linux/).
* Variable Name: `QLIC`
* Variable Value: `/user/path/to/folder`

#### Commercial Evaluation License

The following steps outline the process by which a user can gain access to an install a kdb Insights license which provides access to PyKX

1. Contact you KX sales representative or sales@kx.com requesting a trial license for PyKX evaluation. Alternately apply through https://kx.com/book-demo.
2. On receipt of an email from KX providing access to your license download this file and save to a secure location on your computer.
3. Set an environment variable on your computer pointing to the folder containing the license file (instructions for setting environment variables on PyKX supported operating systems can be found [here](https://chlee.co/how-to-setup-environment-variables-for-windows-mac-and-linux/).
* Variable Name: `QLIC`
* Variable Value: `/user/path/to/folder`

__Note:__ PyKX will not operate with a vanilla or legacy kdb+ license which does not have access to specific feature flags embedded within the license. In the absence of a license with appropriate feature flags PyKX will fail to initialise with full feature functionality.
Full instructions on license installation are covered on the [Installing page](docs/getting-started/installing.md).

### Supported Environments

KX only officially supports versions of PyKX built by KX, i.e. versions of PyKX installed from wheel files. Support for user-built installations of PyKX (e.g. built from the source distribution) is only provided on a best-effort basis. Currently, PyKX provides wheels for the following environments:
KX only officially supports versions of KDB-X Python built by KX, i.e. versions of KDB-X Python installed from wheel files. Support for user-built installations of KDB-X Python (e.g. built from the source distribution) is only provided on a best-effort basis. Currently, KDB-X Python provides wheels for the following environments:

- Linux (`manylinux_2_17_x86_64`) with CPython 3.8-3.11
- macOS (`macosx_10_10_x86_64`) with CPython 3.8-3.11
- Windows (`win_amd64`) with CPython 3.8-3.11
- Linux (`manylinux2014_x86_64`, `manylinux2014_aarch64`) with CPython 3.9-3.14
- macOS (`macosx_10_15_x86_64`, `macosx_10_15_arm64`) with CPython 3.9-3.14
- Windows (`win_amd64`) with CPython 3.9-3.14

### Dependencies

#### Python Dependencies

PyKX depends on the following third-party Python packages:
KDB-X Python depends on the following third-party Python packages:

- `pandas>=1.2, <2.0; python_version=='3.8'`
- `pandas>=1.2, <3.0; python_version>'3.8'`
- `pandas>=1.2; python_version>'3.8'`
- `numpy>=1.22; python_version<'3.11'`
- `numpy>=1.23; python_version=='3.11'`
- `numpy>=1.26; python_version>'3.11'`
Expand All @@ -103,13 +84,13 @@ PyKX depends on the following third-party Python packages:
- `dill>=0.2.0`
- `requests>=2.25.0`

They are installed automatically by `pip` when PyKX is installed.
They are installed automatically by `pip` when KDB-X Python is installed.

PyKX also has an optional Python dependency of `pyarrow>=3.0.0`, which can be included by installing the `pyarrow` extra, e.g. `pip install pykx[pyarrow]`
KDB-X Python also has an optional Python dependency of `pyarrow>=3.0.0`, which can be included by installing the `pyarrow` extra, e.g. `pip install pykx[pyarrow]`

When using PyKX with KX Dashboards users will be required to install `ast2json~=0.3` this can be installed using the `dashboards` extra, e.g. `pip install pykx[dashboards]`
When using KDB-X Python with KX Dashboards users will be required to install `ast2json~=0.3` this can be installed using the `dashboards` extra, e.g. `pip install pykx[dashboards]`

When using PyKX Streaming users may require the ability to stop processes initialized in a now unavailable process to facilitate this PyKX can make use of `psutil` this can be installed using the `streaming` extra, e.g. `pip install pykx[streaming]`
When using KDB-X Python Streaming users may require the ability to stop processes initialized in a now unavailable process to facilitate this KDB-X Python can make use of `psutil` this can be installed using the `streaming` extra, e.g. `pip install pykx[streaming]`

When using Streamlit users will be required to install `streamlit~=1.28` this can be installed using the `streamlit` extra, e.g. `pip install pykx[streamlit]`

Expand Down Expand Up @@ -162,16 +143,16 @@ python3 -m venv pykx-dev
source pykx-dev/bin/activate
```

Build and install PyKX:
Build and install KDB-X Python:

```bash
cd pykx
pip3 install -U '.[all]'
```

To run PyKX in licensed mode ensure to follow the steps to receive a [Personal Evaluation License](https://code.kx.com/pykx/getting-started/installing.html#personal-evaluation-license)
To run in licensed mode see [here](docs/getting-started/installing.md).

Now you can run/test PyKX:
Now you can run/test KDB-X Python:

```bash
(pykx-dev) /data/pykx$ python
Expand All @@ -198,7 +179,7 @@ export QHOME=/location/of/your/q #q needs QHOME available
python -m pytest -vvv -n 0 --no-cov --junitxml=report.xml
```

## PyKX Licenses
## KDB-X Python Licenses

This work is dual licensed under [Apache 2.0](https://code.kx.com/pykx/license.html#apache-2-license) and the [Software License for q.so](https://code.kx.com/pykx/license.html#qso-license) and users are required to abide by the terms of both licenses in their entirety.

Expand Down
1 change: 0 additions & 1 deletion conda-recipe/conda_build_config.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
python:
- 3.8
- 3.9
- 3.10
- 3.11
Expand Down
10 changes: 3 additions & 7 deletions conda-recipe/meta.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,6 @@ requirements:
- numpy==2.0.* # [py==311]
- numpy==2.0.* # [py==310]
- numpy==2.0.* # [py==39]
- numpy==1.22.* # [py==38]
- numpy==1.20.* # [py==37]
- tomli>=2.0.1
- wheel>=0.36
- sysroot_linux-64 ==2.17 # [linux and py<314]
Expand All @@ -37,12 +35,10 @@ requirements:

run:
- python
- numpy>=1.20 # [py==37]
- numpy>=1.22 # [py>37 , py<314]
- numpy>=1.22 # [py>38 , py<314]
- numpy>=2.3.0 # [py==314]
- pandas>=1.2, <3.0 # [py>38 , py<314]
- pandas>=2.3.3, <3.0 # [py==314]
- pandas<2.0 # [py==38]
- pandas>=1.2 # [py>38 , py<314]
- pandas>=2.3.3 # [py==314]
- pyarrow>=3.0.0
- pytz>=2022.1
- toml>=0.10.2
Expand Down
2 changes: 1 addition & 1 deletion custom_theme/main.html
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@

delightedThumbsP.survey({
properties: { // extra context (optional)
product: "PyKX"
product: "KDB-X Python"
}
});
</script>
Expand Down
2 changes: 1 addition & 1 deletion custom_theme/partials/header.html.tpl
Original file line number Diff line number Diff line change
Expand Up @@ -126,5 +126,5 @@
}
</style>
<body>
<button class="open-button md-typeset" onclick="window.location.href='https://kx.com/slack'"><span class="twemoji"><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 512 512"><path d="M256 32C114.6 32 0 125.1 0 240c0 47.6 19.9 91.2 52.9 126.3C38 405.7 7 439.1 6.5 439.5c-6.6 7-8.4 17.2-4.6 26S14.4 480 24 480c61.5 0 110-25.7 139.1-46.3C192 442.8 223.2 448 256 448c141.4 0 256-93.1 256-208S397.4 32 256 32zm0 368c-26.7 0-53.1-4.1-78.4-12.1l-22.7-7.2-19.5 13.8c-14.3 10.1-33.9 21.4-57.5 29 7.3-12.1 14.4-25.7 19.9-40.2l10.6-28.1-20.6-21.8C69.7 314.1 48 282.2 48 240c0-88.2 93.3-160 208-160s208 71.8 208 160-93.3 160-208 160z"></path></svg></span>&nbsp;Ask a question</button>
<button class="open-button md-typeset" onclick="window.location.href='https://kx.com/slack'"><span class="twemoji"><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 512 512"><path d="M256 32C114.6 32 0 125.1 0 240c0 47.6 19.9 91.2 52.9 126.3C38 405.7 7 439.1 6.5 439.5c-6.6 7-8.4 17.2-4.6 26S14.4 480 24 480c61.5 0 110-25.7 139.1-46.3C192 442.8 223.2 448 256 448c141.4 0 256-93.1 256-208S397.4 32 256 32zm0 368c-26.7 0-53.1-4.1-78.4-12.1l-22.7-7.2-19.5 13.8c-14.3 10.1-33.9 21.4-57.5 29 7.3-12.1 14.4-25.7 19.9-40.2l10.6-28.1-20.6-21.8C69.7 314.1 48 282.2 48 240c0-88.2 93.3-160 208-160s208 71.8 208 160-93.3 160-208 160z"></path></svg></span>&nbsp;Ask a question</button>
</body>
2 changes: 1 addition & 1 deletion docs/api/columns.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# PyKX Column Objects
# KDB-X Python Column Objects

::: pykx.wrappers.Column

Expand Down
2 changes: 1 addition & 1 deletion docs/api/compress.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: Compression and encryption
description: API reference page for using kdb+ compression and encryption through PyKX
description: API reference page for using kdb+ compression and encryption through KDB-X Python
author: KX Systems
date: September 2024
tags: compression, encryption, dare
Expand Down
2 changes: 1 addition & 1 deletion docs/api/db.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: DB Admin
description: API reference page for managing kdb+ databases with PyKX
description: API reference page for managing kdb+ databases with KDB-X Python
author: KX Systems
date: September 2024
tags: admin, management
Expand Down
Loading
Loading