Spaces:
Running
Running
Docs for basic usage, workspaces.
Browse files
README.md
CHANGED
@@ -15,6 +15,8 @@ The two tools offer similar functionality, but are not compatible.
|
|
15 |
This version runs on GPU clusters instead of Hadoop clusters.
|
16 |
It targets CUDA instead of Apache Spark. It is much more extensible.
|
17 |
|
|
|
|
|
18 |
## Structure
|
19 |
|
20 |
- `lynxkite-core`: Core types and utilities. Depend on this lightweight package if you are writing LynxKite plugins.
|
|
|
15 |
This version runs on GPU clusters instead of Hadoop clusters.
|
16 |
It targets CUDA instead of Apache Spark. It is much more extensible.
|
17 |
|
18 |
+
Check out our [**online demo**](https://lynx-analytics-lynxkite.hf.space/).
|
19 |
+
|
20 |
## Structure
|
21 |
|
22 |
- `lynxkite-core`: Core types and utilities. Depend on this lightweight package if you are writing LynxKite plugins.
|
docs/guides/analytics.md
ADDED
@@ -0,0 +1,82 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
# Graph analytics & data science
|
2 |
+
|
3 |
+
Install LynxKite with the graph analytics package:
|
4 |
+
|
5 |
+
```bash
|
6 |
+
pip install lynxkite lynxkite-graph-analytics
|
7 |
+
```
|
8 |
+
|
9 |
+
Run LynxKite in your data directory:
|
10 |
+
|
11 |
+
```bash
|
12 |
+
cd lynxkite-data
|
13 |
+
lynxkite
|
14 |
+
```
|
15 |
+
|
16 |
+
LynxKite by default runs on port 8000, so you can access it in your browser at
|
17 |
+
[http://localhost:8000](http://localhost:8000).
|
18 |
+
To run it on a different port, set the `PORT` environment variable (e.g., `PORT=8080 lynxkite`).
|
19 |
+
|
20 |
+
## Using a CUDA GPU
|
21 |
+
|
22 |
+
To make full use of your GPU, install the `lynxkite-graph-analytics` package with GPU support.
|
23 |
+
|
24 |
+
```bash
|
25 |
+
pip install lynxkite 'lynxkite-graph-analytics[gpu]'
|
26 |
+
```
|
27 |
+
|
28 |
+
And start it with the cuGraph backend for NetworkX:
|
29 |
+
|
30 |
+
```bash
|
31 |
+
NX_CUGRAPH_AUTOCONFIG=true lynxkite
|
32 |
+
```
|
33 |
+
|
34 |
+
## Directory browser
|
35 |
+
|
36 |
+
When you open the LynxKite web interface, you arrive in the directory browser. You see
|
37 |
+
the files and directories in your data directory — if you just created it, it will be empty.
|
38 |
+
|
39 |
+
You can create workspaces, [code files](plugins.md), and folders in the directory browser.
|
40 |
+
|
41 |
+
## Workspaces
|
42 |
+
|
43 |
+
A LynxKite workspace is the place where you build a data science pipeline.
|
44 |
+
Pipelines are built from boxes, which have inputs and outputs that can be connected to each other.
|
45 |
+
|
46 |
+
To place a box, click anywhere in the workspace. This opens a search menu where you can
|
47 |
+
find the box you want to add.
|
48 |
+
|
49 |
+
## Importing your data
|
50 |
+
|
51 |
+
To import CSV, Parquet, JSON, and Excel files, you can simply drag and drop them into the LynxKite workspace.
|
52 |
+
This will upload the file to the server and add an "Import file" box to the workspace.
|
53 |
+
|
54 |
+
You can also create "Import file" boxes manually and type the path to the file.
|
55 |
+
You can either use an absolute path, or a relative path from the data directory.
|
56 |
+
(Where you started LynxKite.)
|
57 |
+
|
58 |
+
## Neural network design
|
59 |
+
|
60 |
+
The graph analytics package includes two environments, _"LynxKite Graph Analytics"_, and _"PyTorch model"_.
|
61 |
+
Use the dropdown in the top right corner to switch to the "PyTorch model" environment.
|
62 |
+
This environment allows you to define neural network architectures visually.
|
63 |
+
|
64 |
+
The important parts of a neural network definition are:
|
65 |
+
|
66 |
+
- **Inputs**: These boxes stand for the inputs. You will connect them to actual data in the workspace that
|
67 |
+
uses this model.
|
68 |
+
- **Layers**: The heart of the model. Use the _"Repeat"_ box looping back from the output of a layer to the
|
69 |
+
input of an earlier layer to repeat a set of layers.
|
70 |
+
- **Outputs**: These boxes mark the place in the data flow that holds the predictions of the model.
|
71 |
+
- **Loss**: Build the loss computation after the output box. This part is only used during training.
|
72 |
+
- **Optimizer**: The result of the loss computation goes into the optimizer. Training is partially configured
|
73 |
+
in the optimizer box, partially in the training box in the workspace that uses the model.
|
74 |
+
|
75 |
+
Once the model is defined, you can use it in other workspaces.
|
76 |
+
|
77 |
+
- Load it with the _"Define model"_ box.
|
78 |
+
- Train it with the _"Train model"_ box.
|
79 |
+
- Generate predictions with the _"Model inference"_ box.
|
80 |
+
|
81 |
+
See the [_Model definition_ and _Model use_ workspaces](https://github.com/lynxkite/lynxkite-2000/tree/main/examples)
|
82 |
+
for a practical example.
|
docs/{usage → guides}/plugins.md
RENAMED
File without changes
|
docs/{usage → guides}/quickstart.md
RENAMED
@@ -23,3 +23,5 @@ lynxkite
|
|
23 |
```
|
24 |
|
25 |
Open [http://localhost:8000/](http://localhost:8000/) in your browser.
|
|
|
|
|
|
23 |
```
|
24 |
|
25 |
Open [http://localhost:8000/](http://localhost:8000/) in your browser.
|
26 |
+
|
27 |
+
Find example workspaces in the [`examples` folder](https://github.com/lynxkite/lynxkite-2000/tree/main/examples).
|
lynxkite-core/src/lynxkite/core/executors/one_by_one.py
CHANGED
@@ -18,8 +18,8 @@ class Context(ops.BaseConfig):
|
|
18 |
Attributes:
|
19 |
node: The workspace node that this context is associated with.
|
20 |
last_result: The last result produced by the operation.
|
21 |
-
|
22 |
-
|
23 |
"""
|
24 |
|
25 |
node: workspace.WorkspaceNode
|
|
|
18 |
Attributes:
|
19 |
node: The workspace node that this context is associated with.
|
20 |
last_result: The last result produced by the operation.
|
21 |
+
This can be used to incrementally build a result, when the operation
|
22 |
+
is executed for multiple items.
|
23 |
"""
|
24 |
|
25 |
node: workspace.WorkspaceNode
|
lynxkite-graph-analytics/pyproject.toml
CHANGED
@@ -19,6 +19,10 @@ dependencies = [
|
|
19 |
"torch-geometric>=2.6.1",
|
20 |
"umap-learn>=0.5.7",
|
21 |
]
|
|
|
|
|
|
|
|
|
22 |
|
23 |
[project.optional-dependencies]
|
24 |
dev = [
|
|
|
19 |
"torch-geometric>=2.6.1",
|
20 |
"umap-learn>=0.5.7",
|
21 |
]
|
22 |
+
classifiers = ["License :: OSI Approved :: GNU Affero General Public License v3 or later (AGPLv3+)"]
|
23 |
+
|
24 |
+
[project.urls]
|
25 |
+
Homepage = "https://github.com/lynxkite/lynxkite-2000/"
|
26 |
|
27 |
[project.optional-dependencies]
|
28 |
dev = [
|
mkdocs.yml
CHANGED
@@ -7,9 +7,10 @@ nav:
|
|
7 |
- Home:
|
8 |
- Overview: index.md
|
9 |
- License: license.md
|
10 |
-
-
|
11 |
-
-
|
12 |
-
-
|
|
|
13 |
- API reference:
|
14 |
- LynxKite Core:
|
15 |
- reference/lynxkite-core/ops.md
|
|
|
7 |
- Home:
|
8 |
- Overview: index.md
|
9 |
- License: license.md
|
10 |
+
- Guides:
|
11 |
+
- guides/quickstart.md
|
12 |
+
- guides/analytics.md
|
13 |
+
- guides/plugins.md
|
14 |
- API reference:
|
15 |
- LynxKite Core:
|
16 |
- reference/lynxkite-core/ops.md
|